... 1 The Identification and Evaluation of the Knowledge and Skills in a Professional Selling Program at a Small University A Capstone Project Presented to the Faculty of the Fred S. Klipsch Educators College Marian University _______________________________________ In Partial Fulfillment of the Requirements for the Degree Doctor of Education in Organizational Leadership _______________________________________ Lori Rumreich May 26, 2022 2 Fred S. Klipsch Educators College Marian University Indianapolis, Indiana APPROVAL OF THE CAPSTONE PROJECT This capstone project, The Identification and Evaluation of the Knowledge and Skills in a Professional Selling Program at a Small University, has been approved by the Graduate Faculty of the Fred S. Klipsch Educators College in partial fulfillment of the requirements for the degree of Doctor of Education. Chair, Jeffery Kaufman, Ph.D Committee Member Robert Schuttler, Ph.D. Committee Member Robert Kubacki, MBA 3 Table of Contents List of Tables .................................................................................................................................. 8 List of Figures ............................................................................................................................... 10 Executive Summary ...................................................................................................................... 11 Chapter 1: Introduction ................................................................................................................. 13 Background ............................................................................................................................... 13 State of Higher Ed: Enrollment Challenges, Financial Challenges, and Industry Needs ...... 13 Employment for Business Graduates .................................................................................... 14 Sales Education in U.S. Higher Education ............................................................................ 15 Problem of Practice ................................................................................................................... 16 The Status of Sales Education in Business................................................................................ 18 Significance and Purpose of the Project .................................................................................... 19 Significance of the Project ..................................................................................................... 19 Purpose of the Project ............................................................................................................ 20 System Diagnosis ...................................................................................................................... 21 Needs Assessment ................................................................................................................. 21 Stakeholders, their Loyalties, and Interactions to Understand their Positions ...................... 23 Conceptual Framework ............................................................................................................. 23 Research Question ..................................................................................................................... 24 Limitations and Delimitations ............................................................................................... 24 4 Definition of Terms ............................................................................................................... 25 Chapter 2: Literature Review ........................................................................................................ 28 General Background of Sales and Sales Programs ................................................................... 28 Market Trends for the Sales Profession and University Sales Education ............................. 28 Why Students Study Sales ..................................................................................................... 32 Industry Partnerships ............................................................................................................. 33 Sales Program Development ..................................................................................................... 36 How to Run, Manage, and Grow a Sales Program ................................................................ 36 Sales Program Curriculum Development .............................................................................. 39 Expected Knowledge and Skills and Typical Pedagogy in Sales Education ........................ 45 Sales Technology and Ethics in Professional Selling Programs ........................................... 49 Evaluation of Sales Education Programs within Business Schools .......................................... 52 Accrediting Agencies for evaluation Sales Program in Higher Education ........................... 52 Other Examples of Sales Program Evaluation in Higher Education ..................................... 54 Chapter 3: Methodology ............................................................................................................... 58 Methodology Part 1: Benchmarking best practice universities ............................................. 59 Methodology Part 2: Qualitative inquiry for program development ..................................... 60 Methodology Part 3: Quantitative survey of program students ............................................. 62 Methodology Part 4: Program evaluation .............................................................................. 63 5 Sampling ................................................................................................................................ 65 Research Instruments and Data Collection ............................................................................ 65 Data Analysis Plan .................................................................................................................... 66 Part 1: Benchmarking best practice universities.................................................................... 66 Part 2: Qualitative inquiry for program development ........................................................... 66 Part 3: Quantitative survey of program ................................................................................. 67 Part 4: Program evaluation .................................................................................................... 67 Chapter 4: Study Results ............................................................................................................... 70 Sales Program Design and Implementation, Parts 1 through 3 ................................................. 70 Program Design Part 1 Results: Benchmarking Similar, Regional, and Aspirational Programs ................................................................................................................................ 70 Sales Program Design Part 2 Results: Qualitative Inquiry .................................................... 73 Sales Program Curriculum Development and Implementation ............................................. 74 Sales Program Design Part 3 Results: Quantitative Survey of Students ............................... 75 Sales Program Evaluation, Part 4 of the Methodology ............................................................. 86 Level 1 Reaction .................................................................................................................... 87 Level 2 Learning .................................................................................................................... 89 Level 3 Behavior ................................................................................................................... 91 Level 4: Results ..................................................................................................................... 95 Chapter 5: Discussion and Evaluation of the Capstone Project .................................................... 97 6 Summary ................................................................................................................................... 97 Summary of the Project and Significance of the Outcome ................................................... 97 Discussion of the Knowledge and Skill Requirements Quantitative Survey ........................ 99 Discussion of the Program Evaluation Results.................................................................... 101 Limitations ........................................................................................................................... 107 Evaluation of the Intervention ............................................................................................. 107 Implications and Reflection ................................................................................................. 108 Future Research ................................................................................................................... 109 References ................................................................................................................................... 111 Appendix A ................................................................................................................................. 121 USCA Criteria for Membership .............................................................................................. 121 Appendix B ................................................................................................................................. 122 Research Instruments .............................................................................................................. 122 Qualitative Interview Guide ................................................................................................ 122 Survey Instrument for Sales Students .................................................................................. 124 Appendix C ................................................................................................................................. 126 Rubrics .................................................................................................................................... 126 Level 1: Reaction, Student Evaluation of Instruction Questions......................................... 126 Level 2: Learning, Role Play Rubric ................................................................................... 127 7 Level 3: Behavior/Transfer, Supervisor Internship Evaluation ........................................... 129 Appendix D ................................................................................................................................. 130 Display Matrix of Knowledge, Skills, and Pedagogy Findings .............................................. 130 Display Matrix of Knowledge, Skills, and Pedagogy Findings .............................................. 131 Course Framework for Knowledge and Skills ........................................................................ 132 Course Framework with Pedagogy Use .................................................................................. 132 8 List of Tables Table 1: Key Milestones in Sales Education ................................................................................ 30 Table 2: Number of Students Participating in Sales at Universities Offering a Sales Major ....... 30 Table 3: Common Sales Program Types at USCA Member Universities .................................... 42 Table 4: Stetson Universitys Sales Major and Minor Course List .............................................. 44 Table 5: Winona State Universitys Sales Courses for a Minor ................................................... 45 Table 6: Sales Practitioner Skill Inventory Categorized as Critical, Important, and Useful ........ 46 Table 7: Sales Practitioner Pedagogy Inventory Categorized as Critical, Important, and Useful 48 Table 8: Summary of Business School Accreditation Agencies .................................................. 52 Table 9: Mapping Course Components for Experiential Learning and Blooms Taxonomy ....... 55 Table 10: Sales Program Evaluation Model using the Adapted Kirkpatrick Four Levels............ 64 Table 11: Sales Program Characteristics ...................................................................................... 71 Table 12: Knowledge and Skills Outcomes at Comparison Schools ............................................ 72 Table 13: Sales Course Types Offered by Program...................................................................... 73 Table 14: Sampling Data from Student Survey ............................................................................ 76 Table 15: Survey Participant Demographics ................................................................................ 76 Table 16: Sales Courses Compeleted ............................................................................................ 77 Table 17: Competition Experience ............................................................................................... 77 Table 18: Sales Program Outcomes .............................................................................................. 78 Table 19: Sales Program Outcomes: Introductory and Advances Sales Course Comparison ...... 79 Table 20: Mean Results for Selling Skills .................................................................................... 81 Table 21: Mean Results for Self-Management and Customer Relationship Management ........... 83 Table 22: Mean Results for Perceptions of the Selling Profession ............................................... 84 9 Table 23: Mean Results for Perceptions about a Selling Career .................................................. 85 Table 24: Interest in a Sales Career .............................................................................................. 85 Table 25: Mean Results for Importance of Sales Program Features ............................................ 86 Table 26: Student Evaluation of Instruction Results .................................................................... 88 Table 27: Introductory Sales Course; Within Group Role Play Evaluation ................................. 90 Table 28: Advanced Sales Course; Within Group Role Play Evaluation ..................................... 90 Table 29: Introductory and Advanced Course Between Group Final Role Play Comparison ..... 91 Table 30: Advanced Sales End of Course Team Project Evaluation ............................................ 93 Table 31: Negotiation End of Course Project Evaluation ............................................................. 94 Table 32: Sales Internship Supervisor Evaluation ........................................................................ 94 Table 33: Career Outcomes and Comparison ............................................................................... 96 10 List of Figures Figure 1: Conceptual Framework for Sales Program Development.. 24 Figure 2: Map of Universities Offering a Sales Major 31 11 Executive Summary This capstone documented the process of defining, developing, and evaluating a curriculum for professional selling within a business school in a small liberal arts university. Specifically, it addressed the research question: What knowledge and skills do students need in preparation for a professional selling career, and how does implementing these in a formal curriculum improve students' sales competencies? Identification of the required knowledge and skills and the associated new curriculum were developed through secondary research, benchmarking other programs, and qualitative interviews with sales curriculum experts. Competency was defined as the ability to implement the knowledge and skills. The model used to evaluate student competencies throughout this new curriculum was a variation on the four-level Kirkpatrick model of reaction, learning, behavior, and results. Specifically, Level 2, Learning and Level 3, Behavior, were used to measure sales competencies and answer the research question being studied. Level 1, Reaction and Level 4, Results were identified as necessary for a comprehensive review of the new program being implemented, although they did not explicitly address the research question. The newly approved curriculum requires the completion of three, 3-credit hour courses, including an introductory level sales course and an advanced sales course. Students then choose additional courses from a list of options to complete the specialization. The program was defined with three overall learning objectives. These objectives were met through the completion of this new curriculum, including nine core knowledge and skill areas. These nine core knowledge and skill areas were introduced in the required introductory course and reinforced in the advanced level course. Ten additional, more advanced skills were introduced and practiced in the advanced course and elective courses. 12 The new curriculum was implemented during fall 2020, spring 2021, and fall 2021 semesters. Prior to the formal evaluation, students assessed the new curriculum through a survey instrument and their feedback provided direction for adjustments within individual courses. These results showed a high level of satisfaction among the students in the program. The Level 2 and 3 formal evaluation results were obtained at the end of each of the three semesters. A significant increase in sales competency was observed, as measured by (a) within group knowledge and skill development gains using a pretest; post-test role play exercise in the introductory and advanced sales courses; (b) end of course project evaluations in two courses; and (c) internship supervisor evaluations. Additional results of interest to the program design included the Level 4 findings of positive career outcomes among program graduates. Some lessons learned during this project include the importance of 1) using adaptive leadership principles in a projects early stages, especially when there are a variety of collaborators and stakeholders with differing values; 2) communicating needs and constraints to collaborators and stakeholders; 3) anticipating difficulties and accepting limitations within the organization, its structure, and its processes; and 4) developing leadership capacity to effectively manage the outcome while generating excitement to engage with a vision for the future. Keywords: professional selling, curriculum design, Kirkpatrick model, sales competency, sales training, university sales. 13 Chapter 1: Introduction This project identified the knowledge and skills needed by students as part of a new program for professional selling and reviewed the newly established curriculum that delivered the knowledge and skills that were defined. An evaluation plan to measure the effectiveness of the knowledge and skills gained by the students in the program was implemented and the results are presented in this document. This implementation project was performed within the context of the adaptive leadership framework. Background State of Higher Ed: Enrollment Challenges, Financial Challenges, and Industry Needs Higher education has changed, and the pandemic accelerated the trends that were already occurring. Undergraduate enrollment was down 4.4% in the fall of 2020 as compared to fall 2019, with community colleges leading the loss in enrollment, and all institution types seeing declines (Williams, 2020). Leading business journals such as Forbes and the Wall Street Journal routinely publish about the looming financial crisis in higher education and the likely closure of hundreds of colleges and universities in the next ten years (Belkin, 2020; Craig, 2020). These experts point to many factors. Some include the demographic cliff of college-aged students and the resulting lack of bodies available to sustain the number of colleges and universities in operation, the unsustainable tenure structure of universities, untenable prices and growing student debt, and a lack of alignment between what universities teach and what industry requires (Nietzel, 2020). Craig (2020) wrote that addressing student debt, or even free college, is not enough. it may not lead to increased economic opportunity as it omits a fairly important component of the employment equation: the employer (para. 5). Craig went on to write that employers and 14 colleges lack alignment, with the former seeking candidates with specific skill sets, especially digital skills. He recommended a model of higher education where faculty are tasked with building relationships with industry that integrate opportunities for networking and employment throughout a students college career. Supporting Craigs opinion, Belkin (2020) also predicted that private industry will play a key role in this reshaping of the workforce and that for colleges to remain relevant, they must develop closer relationships with companies. Employment for Business Graduates Business education remains a potential growth area for universities. According to the Bureau of Labor Statistics (BLS) Occupational Outlook Handbook (2019), employment opportunities in business fields continue to increase, driven by the strong median annual wage of nearly seventy thousand dollars annually as of May 2019. According to the Handbook, employment in business and financial operations occupations is projected to grow five percent from 2019 to 2029, faster than the average for all occupations, adding about 476,200 new jobs. It also projects that employment in accounting, market research, general management, sales, and supply chain management will grow faster than average. For sales professions specifically, BLS predicted that professional sales employment would remain strong through 2029, with insurance sales growing at a five percent rate 2019 2029 and a mean annual income of $50,940, and financial services sales positions growing at the market average, four percent, and a median pay of $62,270 per year. U.S. News and World Report, in the Best Job Rankings 2020 report, stated a median salary for all types of sales representatives of $58,510. A study by Emsi (2020) used 100 million professional profiles to identify job types among college degree holders. The study reported that among those with a formal degree in 15 business, 18% were employed in sales, 15% in business and financial analysis, 14% in accounting, 9% in marketing, and 6% in human resources. The study also reported that sales was the most common career outcome among all degree holders, including among graduates in humanities, social sciences, and engineering. In the study definitions, sales included technical sales, financial sales, and business to business sales, since these are the sales occupations most relevant to college graduates. The study did not include retail sales employment in the sales category. Research showed that students with a family history in sales tended to show a greater interest in a sales career than did students without this background (Crawford & Lumpkin, 1983; Spillan, Totten, & Chaubey, 2011). Students who studied business also tended to show more interest in a sales career than did students in other majors, suggesting the role that major plays in the selection of sales as a future career (Spillan, Totten, & Ziemnowicz, 2007; Spillan et al., 2011). Sales Education in U.S. Higher Education According to the Sales Education Foundation (SEF), the strong market demand, employment growth, compensation levels, and its position as a springboard to more senior management roles in marketing, finance and other functions have led to increased interest and job placement in professional selling careers among college graduates (SEF Key Statistics, 2020). This has in turn led to an increase in interest among employers in connecting with universities and has resulted in the recent growth in sales courses within the marketing curriculum at the university level (SEF Annual, 2020; Spiller, Kim, & Aitken, 2019). In 2002, the University Sales Center Alliance (USCA) was born in response to this increased market demand. Nine universities collaborated to form this organization with the mission, To advance 16 the selling profession through setting and monitoring sales program standards, sharing best practices, enhancing sales curricula, and preparing students for a career in sales (USCA, 2020, About Us section, para. 2). This organization has continued to gain prestige and today represents 35 full members and 23 associate members (USCA, 2020). While the USCA website provides direction on requirements for sanctioned sales centers within universities, a review of their member list shows that they primarily serve large universities with significant resources and dedicated, tenured sales faculty (USCA, 2020). A comparison of smaller universities and colleges to these USCA members shows that smaller programs are unlikely to possess the faculty and resources needed to meet the USCA requirements (SEF Annual, 2020). At the same time, smaller universities may be uniquely positioned to bring value to sales education for students and employers if they are able to deliver on a curriculum that prepares students with the required knowledge and skills to be successful in professional selling roles. Hallmarks of sales education include small class sizes, significant one-on-one interaction with faculty using role plays and other individual and small group practice, and close industry partnerships (Forbes, Loe, Patterson, & Erffmeyer, 2014; Spiller, Kim, & Aitken, 2019; Weilbaker, 2004). Most small universities already support small class sizes and the resulting closer relationships with students. In addition, smaller schools may be able to act more nimbly when it comes to rapid technological change as long as they are able to align their curriculum and course outcomes with industry needs. Problem of Practice There is a need for professional sales representatives in the labor force (BLS, 2019; Manpower Group, 2018; SEF Key Statistics, 2020). This need continues to grow due to the limited number of university professional selling programs in place to help prepare students for 17 professional selling roles upon graduation (Bolander, Bonney, & Satornino, 2014; Fogel, Hoffmeister, Rocco, & Strunk, 2012; Spiller, et al., 2019; USCA, 2020). The challenge of preparing students for the workplace is not a new phenomenon in business education, but what is new are the differences occurring in sales education. Namely, the lagging sales student pipeline, the accelerated rate and scope of change in the knowledge and skills needed to prepare students for a selling role in the marketplace, and the lack of research available to assess that the required knowledge and skills have been attained by new graduates (Peltier, Cummins, Pomirleanu, Cross, & Simon, 2014; Spiller, Kim, & Aitken, 2019). Although many programs claim successful sales student outcomes, how they identified current and emerging knowledge and skill requirements and incorporated them in their programs, and how they evaluated the efficacy of their programs has not been well documented. Although a modest body of research exists on sales education pedagogies, scholars in the field have identified the need for more research on sales education from smaller schools, identification of an ideal sales curriculum, and, for more research studies that use direct measures of performanceto advance the sales education field (Spiller et al., 2019, p. 229). To address the need for professional sales representatives who are prepared for professional selling roles at graduation, understanding the current knowledge and skills needed to prepare them, implementing these competencies in a formal sales curriculum, and evaluating student performance on these competencies is critical. Faculty at the university being studied for this project had identified a need for new programs with both student and market interest to ensure future student enrollment and job placement success. Internally, students had shown interest in professional selling coursework, with approximately 60% of marketing and 20% of management business graduates from the past 18 two years finding employment in a sales role upon graduation (Job Placement Statistics, 2019). However, the business school administration and faculty of these graduates have information that not all graduates had been successful in their first year, suggesting that the problem may relate to their academic preparation. The Status of Sales Education in Business To maintain current enrollment levels and financial returns in the face of the dwindling market opportunity, business schools must remain focused on attractive programs that will support recruitment efforts while satisfying employer demand for highly qualified graduates (Craig, 2020). To implement a successful new sales program, identification of business partner needs within the local geographic market under study for this project was needed. An understanding of student knowledge and skills needed for new sales graduates was critical to achieving strong partnerships and their associated financial support to maintain the program (Forbes et al., 2014). Finally, although there was general agreement on the need for a sales program, there are potential issues to be resolved related to program management and control, budget authority, and the partner management process. These issues will be further explored in the System Diagnosis section of this paper. Mission and culture are advantages within the business school being studied for this project. The school serves a population with higher than average numbers of minority students and economically disadvantaged students as compared to universities overall (Office of Institutional Research, 2020). Sales careers, with strong market demand, high entry-level salaries, and promotion potential within businesses, align well with the business school mission to serve these students and develop business leaders. While the business school is a teaching 19 institution and lacks the AACSB research-oriented accreditation typical of top sales schools, it has a culture based on a hands-on, experiential curriculum with significant industry professional interaction and faculty who have industry experience in their fields. These factors, along with the previously outlined market demand and growth, expose an opportunity for educational offerings like professional sales. Until 2019, the business school in this study offered only a single introductory sales course. A second sales course was introduced in late 2019, Advanced Selling. It resulted in positive student feedback, internship and employment opportunities for participating students, and early success in national sales competition events. A third course, Negotiation, was piloted in the spring of 2020. The business school also enjoyed an advantage with its nascent sales program, as the only university in the immediate geographic area with a sales program. A concern with the current sales offerings was that they were piloted with little input from potential partner businesses. Students who pursue careers in sales found themselves interviewing alongside students from schools with strong national rankings, brand recognition, and employer agreements, such as USCA schools. To compete with these schools, business school students must be well prepared in knowledge and skills and must make connections with potential employers during their time in school. Significance and Purpose of the Project Significance of the Project Professional selling was identified as a program that would prepare students to make an impact in a professional selling role upon graduation (Bolander, Bonney, & Satornino, 2014). This study addressed a local problem by focusing specifically on the development and evaluation of a professional selling program for students in the university under study. This project was 20 unique because it addressed an under-researched area of higher education within business schools (Bolander et al., 2014; Cummins et al., 2013; Deeter-Schmelz & Kennedy, 2011; Peltier et al., 2014; Spiller et al., 2019) in a high employment growth area (BLS, 2019; Manpower Group, 2018; SEF Key Statistics, 2020). The results of this study provided much-needed insights for the study university into the curriculum and pedagogy of a new sales program, along with establishing a clear evaluation process. Insights from this study aided the business school in helping students to succeed academically, in their internship and job searches, and after graduation, thus supporting enrollment goals and student graduation rates, industry funding, and business school reputation (Bolander et al., 2014; Cummins et al., 2013; HR Daily Advisor, 2018; Lilly & Stanley, 2016; Newberry & Collins, 2015; SEF Key Statistics, 2020; Weilbaker, 2004). Further, higher education has long been a tool used to address social inequality. This study also supported the successful attainment of professional sales employment among underrepresented student groups and increased diversity in the types of individuals gaining these positions. Purpose of the Project The purpose of this study was to identify the knowledge and skills needed to meet the educational objectives and market needs in the design and delivery of a professional selling program at the university under study and create and implement an evaluation framework for the measurement of program effectiveness. Although the business school under study was traditionally a strong revenue producer for the university, new student enrollment overall had been stagnant for several years and future enrollment projections nationally portend declines (Anderson & Douglas-Gabriel, 2022). Even with strong revenues, the business school had difficulty in meeting growing budget needs and 21 had not increased the number of faculty, staff, or administration in over ten years due to budget pressures. If, as projected nationally, enrollment is flat or even declines, consequences for the business school and the university could be severe. For these reasons, identifying new programs like sales, with strong employer demand and potential industry funding, and then designing a curriculum to meet these needs, presented opportunities for growing student enrollment and increasing external funding for the school. A program in professional selling was well aligned with the goals of strong employment prospects, good opportunities for industry funding and support, and high student interest. However, to be successful, the knowledge and skills required by graduates must be well understood, implemented, and measured effectively in the curriculum. The lack of research in these areas, specifically in schools similar in size and geographic location to the school under study, formed the argument for this project. System Diagnosis Adaptive leadership concepts were applied to (a) assess the needs of the organization; (b) understand stakeholder positions and document interventions; and, (c) summarize the learning that occurred to address the complex challenge that was presented by this project. Needs Assessment The business school marketing program has had success over the past 12 years in marketing-oriented competitions but had little focus on sales courses or competitions. The competition success resulted in positive public relations for the school. Student feedback was also positive, with many students requesting additional opportunities to include a wider range of competition options. 22 To identify new opportunities, a review of competition opportunities was performed. Several ideas emerged, such as sales competitions offered through the American Marketing Association (AMA), as well as university-sponsored sales competitions. With only a single sales course being offered in the curriculum before the fall of 2019, further study was done to identify if sales competitions would be appropriate. Job placement statistics for recent graduates were reviewed. It was determined that many business graduates were being employed in some type of sales role as their first professional position after graduation. To test the viability of pursuing additions to the sales curriculum, a faculty member invited a small group of students to participate in a regional beginners level sales competition. Student feedback was tremendously positive. A SWOT analysis was performed to help evaluate the abilities and potential for success of the business school to create, launch, and manage a new program in professional selling. Strengths of the school were that it was relatively small, with good communication among the faculty and administration. The school had a history of successfully implementing new courses and even new major programs of study. Another strength was the experiential nature of a sales program. The hands-on format aligned well with the current culture in the business school. Space was not expected to be an issue since the capacity for growth already existed. A potential weakness, or concern, was the financial resources that might be needed to run a new program, including new faculty. Opportunities were that a new program in sales would enjoy a competitive advantage since there were no universities in the local area offering sales majors, minors, or other programs of study devoted to sales. Another opportunity was employer demand and the willingness of local business partners to support a program financially. Strengthening 23 business partnerships provides opportunities for student internships and job placement. A threat to a new program was that another local university might decide to offer something similar. Stakeholders, their Loyalties, and Interactions to Understand their Positions The adaptive leadership process (Heifetz, Grashow, & Linsky, 2009) was used to understand the formal and informal dynamics within the organization. Stakeholders were identified and multiple interactions occurred over a six month period to clarify the needs and goals of each individual or group. Formal meetings and informal discussions occurred with the business school dean, faculty, adjunct faculty, university advancement staff, and external business professionals who expressed an interest in recruiting from or donating to a future selling program. University policies were also examined to ensure adherence to all requirements for the new program and course development and approval. Conceptual Framework Using the adaptive leadership process for system diagnosis allowed a more in-depth understanding of the needs of the organization, the marketplace, and the stakeholders. It resulted in a framework for how to construct a new program in professional selling. The research and learning from the process were then used to form the conceptual framework, as shown in Figure 1. A combination of adaptive and technical processes were used to implement the solution of desired knowledge and skills. This ensured all faculty teaching within the new program were part of program design and achieved agreement on the framework and implementation plan. Both adaptive leadership and technical processes were used as part of the ongoing program evaluation. 24 Figure 1 Conceptual Framework for Sales Program Development Research Question Based on the information presented, the project research question was stated as: What knowledge and skills do students need in preparation for a professional selling career, and how does implementing these in a formal curriculum improve students' sales competencies? Limitations and Delimitations Some limitations on this studys evaluation methods included one interview being held by telephone due to Covid restrictions, obtaining adequate contact information for the collection of outcomes measures among a small number of graduates, and the ongoing changes to university processes, people, and organizational structure during the project timeframe. Moving forward, limitations regarding running this program are that the business school is small and has limited faculty and staff available to assist if the program continues to grow. Faculty turnover could also impact program viability. 25 A delimitation for the program evaluation was the small student population being studied. This study required the measurement of unique populations of current and recently graduated students, with small sample sizes, indicative of case studies. Another delimitation was that this analysis included only a single institution, the university in this study, with a single student population of sales program students, and no opportunity for a comparison of outcomes with other new programs. Definition of Terms a. Adaptive Leadership: a practical leadership framework that helps individuals and organizations adapt to changing environments and effectively respond to recurring problems. It is the activity of mobilizing people to tackle tough challenges and do the adaptive work necessary to make progress and thrive. Research by Heifetz, Grahsow, and Linsky (2009) at Harvard University formed the basis for adaptive leadership. b. Benchmarking: the practice of comparing processes and performance metrics to best practices from other organizations, schools, or departments. c. B2B: The term business-to-business (B2B) refers to the process of selling the goods and services a business makes to another business, either for their own use or for resale to consumers. d. B2C: The term business-to-consumer (B2C) refers to the process of selling products and services directly between a business and consumers who are the end-users of its goods or services. e. Coaching: Coaching is a development or training technique typically used for executives or new managers/leaders. It often involves a professional coach or external consultant who works with employees to develop their skills and achieve business goals. Coaches 26 support employees by understanding their personal and career goals and providing feedback and direction to facilitate performance improvement (Training Industry, 2020). f. Executive in Residence: experts from industry, such as a retired executive, who lend their specialized knowledge and experience to organizations on a temporary or part-time basis. g. Institutional Advancement: a division or department of the institution focused on the improvement of the entity, primarily involved in fundraising activities. h. KSAs: Knowledge, skills and abilities (KSA) is a term referring to the qualifications that an individual needs to perform a specific job. Usually, an employer will create the KSA as a guideline to seek out the most qualified applicants for a position (Training Industry, 2020). i. Major/Minor/Specialization: For a Bachelors degree, a major is a primary focus of study and a minor is a secondary focus of study. At the university in this study, a minor typically requires 18 hours of study. A specialization or concentration differs between universities. At the university in this study, a specialization typically requires 12 credit hours of study. Some universities use the term concentration as an alternative. j. Mentoring: Mentoring is a type of training in which a more experienced employee (the mentor) provides expertise, support and guidance to a less experienced employee (the mentee), in an effort to advance his or her career growth and success in the workplace (Training Industry, 2020). k. Part-time or Adjunct Faculty: An adjunct position is a part-time job. Adjuncts are limited in the number of courses taught, and the compensation is on a per course basis. l. SEF: The Sales Education Foundation, a non-profit organization that promotes the benefits of university sales education both in academia and the corporate world. 27 m. Stakeholder: People affected by the challenge and in need of a solution, sometimes called front-line stakeholder. n. Student Coach: A type of coaching using peer feedback rather than an industry professional. Used to build skills in both the student being coached and the student coach. o. Top Box Score: The top box score used in this study is the sum of percentages for the top two points, such as strongly agree and agree, on a five-point scale. p. USCA: The University Sales Center Alliance is a group of 35 member and 23 associate member universities who meet specific criteria to join the alliance. All member universities have formal Sales Centers with a faculty director. The USCA connects marketing and sales faculty members who advocate for the continuing advancement of the sales profession through teaching, research and outreach (USCA, 2020). 28 Chapter 2: Literature Review The literature review was undertaken to assist in identification of the knowledge and skills needed by students to build sales competency, along with gathering information to support the design and development of a new professional selling program. General Background of Sales and Sales Programs Market Trends for the Sales Profession and University Sales Education Sales and marketing employment is projected to remain strong through 2029, yet sales and marketing positions are among the most challenging to fill (BLS, 2020; Manpower Group, 2018). The BLS Occupational Outlook Handbook provides comparative data across occupations and can help inform data-driven decision making about the current and future demand for a job category or even specific job types. The 2019 data indicated that a sales manager position requires a bachelors degree and that growth for this job is at 4% through 2029. This information lent credence to the argument that a sales career is more than vocational training and is a legitimate academic area of study. More specific categories such as insurance sales, advertising sales, and sales engineers, defined in the BLS as including B2B sales, which is what most academic programs teach, also showed strong rates of job growth through 2029. At the same time, CSO Insights, the research division of Miller Heiman Group, a sales training and consultancy firm, reported that only 16% of sales leaders believe that they have the talent they need to succeed in the future (HR Daily Advisor, 2018). In 2012, author and sales expert Daniel Pink, in his book, To Sell is Human, claimed that the number of salespeople in the United States outnumbers the federal government workforce by five to one. He also cited increasing demand and high turnover rates for salespeople that continue to fuel high sales employment rates. 29 This information lent strong support to the viability of a sales program and its ability to support university enrollment goals through 2029. Further confirmation for this opportunity was found by examining key milestones in higher education related to sales education, as presented in Table 1. These developments continued to spur interest in sales education and provided legitimacy to the field. Growth of sales programs at the university level, although gaining slow momentum among large universities, has lagged behind other business areas in terms of new program growth at both the undergraduate and MBA levels (Cespedes & Weinfurter, 2016). They reported that only about 150 colleges and universities in the U.S. offered some type of sales program. Of those, 31 offered a major in sales. Their research revealed that of those schools with a major, 21 were USCA members and 23 were AACSB accredited. Of the 17 without AACSB accreditation, two held IACBE accreditation, and the remainder did not report any specialized business school accreditation (Cespedes & Weinfurter, 2016; SEF Annual, 2020). Using data reported in the SEF Annual (2020), the number of students participating in the sales program at schools offering a sales major is shown in summary form in Table 2. Student numbers range from a low of 15 at Hodges University in Naples, Florida, an IACBE accredited university, to a high of 500 students at both Florida State University in Tallahassee and Weber State University in Ogden, Utah. Both of these schools are members of USCA. Figure 2 presents a visual representation of the location of all 31 universities offering a sales major (SEF Annual, 2020). 30 Table 1 Key Milestones in Sales Education Year 1980 1984 Key Milestone Journal of Personal Selling & Sales Management was created American Marketing Association (AMA) sponsored the Selling and Sales Management Consortium 1985 The first university sales center (Center for Professional Selling) to open at Baylor University 1989 The second university sales center to open at Northern Illinois University 1999 First National Collegiate Sales Competition (NCSC) held at Kennesaw State University by the Coles College of Business Center for Professional Selling. It is the longest running university sales role play competition in existence. 2002 University Sales Center Alliance (USCA) was formed by the leaders of the then eight established university Sales Centers or Sales Institutes with the purpose to connect faculty from different universities and to advocate for the continuing advancement of the sales profession through teaching, research, and outreach. 2007 Sales Education Foundation (SEF) was established as a nonprofit organization dedicated to promoting the benefits of university sales education both in academia and the corporate world. They publish the annual Top Sales Universities report. Source: Adapted/Updated from Spiller et al, (2019); NCSC, (2020) Table 2 Number of Students Participating in Sales at Universities Offering a Sales Major Number of Students Number of Universities < 49 50 99 100-199 200-299 300+ Source: SEF Annual (2020) 5 7 5 6 8 31 Figure 2 Map of Universities offering a Sales Major Source: SEF Annual, (2020) The combination of market opportunity, employer demand, support from the USCA, and the associated program growth among large and highly respected university marketing departments combined to create a substantial opportunity for those universities that embraced the challenge of creating a sales program (Weilbaker, 2014). The application of digital technologies and artificial intelligence to sales automation and customer relationship management further supported the need for strong sales research and curriculum at the university level (Spiller, et al., 2019). The rapid changes continuing to occur in digital sales technology have strong implications for universities who must navigate these challenges and find ways to integrate this information into the education of the next generation of sales professionals (Singh, Flaherty, Sohi, Deeter-Schmelz, Habel, Le Meunier-FitzHugh, Malshe, Mullins, & Onyemah, 2019). Finally, research by Bolander et al. (2014) provided convincing evidence that connecting the relationship between sales education and the performance of new graduates in their first sales job was an important step toward confirmation of the value of university sales education programs. Although quantitative studies are limited, Bolanders research demonstrated that 32 salespeople hired from universities with formal sales programs outperformed their non-sales program counterparts after the first year of employment. Sales education has also been shown to have a significant, positive impact on salesperson attitude toward his or her company (Bolander et al., 2014). The implications of these findings were significant for universities that offered or planned to develop sales programs in terms of meeting the needs of employers and leveraging those employer relationships for strong partnerships with the university. Why Students Study Sales While market demand is strong and there was evidence that sales education leads to stronger outcomes in a sales career, many young college students hold a negative perception of sales and lack consideration for studying sales or even taking a sales class (Allen, Kumar, Tarasi, & Wilson, 2014; Barat & Spillen, 2009; Peltier, Cummins, Pomirleanu, Cross, & Simon, 2014). It was therefore important to understand those perceptions and what factors might influence positive perceptions of a sales career. An understanding of what does and does not resonate with these students was needed so that marketing educators could better bridge the gap between students and potential employers. Four areas emerged in the literature as the primary contributors to students intent to pursue a professional selling career. Student perceptions and attitudes were shaped by the sales profession, salespeople, sales knowledge, and sales ethics (Bristow, Gulati, Amyx, & Slack, 2006; Bristow et al., 2011; Karakaya, Quigley & Bingham, 2011; Peltier et al., 2014). Karakaya et al. (2011) reported a significant relationship between a students feelings toward selling and the intent to pursue sales as a career. Positive perceptions and attitudes about the sales profession were found with students who perceived sales jobs as valuable, creative, and having opportunities for growth (Allen et al., 2014). Positive feelings were found among students who perceived sales as fun and exciting while requiring intelligence, confidence, and talent 33 (Karakaya et al., 2011). Allen et al. (2014) also reported that advisors had a significant influence on a students intent to study sales, as well as prior interest in the field. At the same time, it was important to ensure that students were forming realistic expectations about a sales career (Allen et al., 2014; Bush, A. J., Bush, V. D., Oakley, & Cicala, 2014). Research suggests that negative perceptions can be overcome through a variety of interventions. For students already studying some area of business as either a major or minor, an introduction to sales in a principles of marketing course was recommended (Peltier et al., 2014). Peltier et al. (2014) proposed the implementation of an Intent To Pursue Selling Career Scale in the principals of marketing course as a means to motivate interest in professional selling early in students educational careers (p. 18). For students in an introductory sales class, exposing them to practical applications and activities in the classroom, such as case studies, role plays, and interactions with sales professionals through mentoring, job shadowing, guest speakers, and networking events was found to increase the positive perception of a sales career (Allen et al., 2014; Bush et al., 2014; Cepedes & Weinfurter, 2016; Gafford, 2019). The use of these activities, along with sales internship experiences and competitions, supported the findings by Bush et al. (2014) which recommended that educators provide a realistic picture of the expectations and behaviors associated with success on the sales job. This information should also be of value in student recruitment to marketing and sales programs, as well as when assisting prospective college students in deciding what to study. Industry Partnerships Nielson and Cummins (2019) indicated that experiential activities and quality involvement by sales professionals in the academic environment were techniques used to influence intent to study sales and provide realistic expectations about a selling career. They also 34 wrote that in addition to learning about sales from practitioners, students required internships and post-graduation employment. Building relationships with companies as part of sales courses could build the networks that will lead to those opportunities. With regard to financial support, Nielson and Cummins (2019) also stated that industry partners could be a very strong source of needed funding, as well as provide brand building for the university. With these factors in mind, it was important to understand how to identify, create, and manage partnerships with sales professionals and their companies so that these experiential activities, job placement opportunities, and university support can become a reality. SEF (2019) summarized three key recommendations for activating corporate-sales center partnerships: Ensure a mutually beneficial strategy and equity in the partnership Maintain an open dialogue allowing for idea sharing Be flexible and collaborative to maximize the impact for university sales programs, corporate partners, and the next generation of sales professionals Fortunately, industry partnerships also provide substantial benefits to the companies who participate (Cepedes & Weinfurter, 2016). Industry professionals have identified the need for involvement in the academic environment as a means to attract top talent to their sales recruitment programs. Weilbaker (2004) suggested that the upward trend in demand for university sales programs was driven primarily by industry, its need to reduce training and other costs associated with the sales force, and overall greater demand for qualified sales talent. This conclusion was supported by the large number of corporations sponsoring university-related sales competitions and other sales program initiatives across the United States. Corporate partners may also have an interest in partnering on research projects that are relevant to issues they face (SEF, 2019). Sales centers must be doing research that is relevant to their corporate 35 partners and that cutting-edge research informs the classroom, according to Deva Rangaranjan, Ph.D., Former Director Center for Professional Selling, Ball State University (SEF, 2019, p. 40). Dr. Charles Ragland, Executive Director, Indiana Universitys Center for Global Sales Leadership, said, We recommend a three-year time horizon for corporate partner engagement (SEF, 2019, p. 40). This timeline is driven by the need for companies to build their brand on campus and develop relationships with program faculty and students. Partners may also wish to be part of the sales programs advisory board as a way to develop faculty and student relationships and steer curriculum (SEF, 2019). According to Lilly and Stanley (2016), these partnerships provided a win-win-win scenario; corporations saw the value, and, as a result of the decline of government funding for higher education, colleges were able to use the additional corporate funding to further their own goals and missions. Their work reinforced the idea that universities with the foresight to develop sales programs and offer specific sales curricula have benefited from higher levels of corporate involvement. Evidence indicated that students saw the benefit in connecting with industry partners to solve business problems, even when the interaction was held virtually (Coco, Crumbly, & Jackson, 2014). Students were more satisfied with courses that had this type of involvement than those that did not and tended to have a more favorable opinion of sales as a discipline than did students who took sales courses without this involvement (Lilly & Stanley, 2016). All stakeholders generally benefited from these offerings due to the tension between the low supply of qualified sales talent and the continuing high demand for qualified entry-level salespeople (Lilly & Stanley, 2016; Weilbaker, 2004). Industry partnerships, while valuable, are not without drawbacks, according to Lilly and Stanley (2016). They summarized the need for additional funding as a result of implementing the 36 activities related to the involvement of industry practitioners. They outlined field trips, networking events, and other sponsorship needs as part of such programs. Faculty time was another potential concern they raised. The focus on behavioral exercises such as role plays required significant coordination with multiple practitioners in terms of logistics and coordination on the part of the faculty involved. Faculty are often tasked with matching students to potential employers, as well as meeting with partners to discuss successes and failures of students hired, and general assessment of benefits to each partner. Sales Program Development How to Run, Manage, and Grow a Sales Program Academic literature on creating and managing a university sales program was very limited (Forbes et al., 2014). Two organizations exist that provide guidance to universities on this topic, the USCA and SEF. Most notably, the USCA (2020) provides guidance on what they feel is essential for the creation and management of a sales program as part of their membership criteria. Although academic literature was limited outside of these two organizations, an early article on the subject referencing the USCA guidelines highlighted the trend to develop subspecialties in marketing departments, such as one in professional sales (Weilbaker, 2004). Weilbaker (2004) reported on how to present the idea of a sales specialization to the marketing department or college of business. His perspective was based on the notion at the time that sales was considered more of a trade school profession. He outlined the transition in sales education and its gains in legitimacy as mirroring the changes that occurred in industry sales organizations. While sales was historically based in retail or B2C areas, by 2004, the rise of B2B selling was clear, and academics, as well as students, began to view the sales profession with more legitimacy. This legitimacy was driven by the need for more logic and analysis in the B2B 37 selling environment and formed the basis for the argument that sales should be considered a viable sub-specialty within university marketing curricula. When making the case for a sales subspecialty, Weilbaker (2004) identified three key stakeholder groups that should be included in a presentation of the benefits of a sales program: students, companies, and the marketing department or business school faculty. SEF published information on the topic based on their research. They provide this research and numerous other resources to educators and industry sales professionals. Another academic publication was in the Journal of Selling. The article provided recommendations on how to establish, run, and grow a sales program. It was authored by five large university sales center directors and summarizes their research with 30 USCA members (Forbes et al., 2014). As an authority on sales programs and as the only organization that certifies sales programs, the USCA provides essential guidance. They have strict membership requirements with specific criteria that must be met for membership in the organization. These criteria are outlined in Appendix A. The USCA requires a 5-year associate member period that includes mentoring by the USCA members while a university works to qualify for full membership (USCA, 2020, Membership section). SEF (2017) provided their recommendations for starting a new program based on research with over 70 university-recognized sales programs. They recommended that a new program launch should be in the form of a sales concentration or specialization, and include a minimum of three courses. They cautioned that a program can take many years to build, with both internal and external hurdles for engaging stakeholders in such an endeavor (SEF, 2019). While both organizations provide guidance on the topic, SEFs guidelines are recommendations, while USCA has set forth requirements based on their membership criteria. For the summary 38 below, the word recommendation was used, however, where USCA is noted, these are membership requirements rather than just recommendations. Both the USCA and SEF recommend a minimum of two faculty in a sales program, with at least one at the academically qualified (AQ) rank. SEF provides further detail on the faculty roles, recommending that one faculty manage research and academic requirements and the other manage corporate relationships, fundraising, and other sales functions of the program (SEF, 2019). Both the USCA and SEF organizations recommend sales courses, including an introductory and a more advanced course. SEF recommends at least one additional specialized course. Also recommended by both organizations are that sales research is performed, programs should be available to students of all majors throughout the university, have external financial support, and hold official recognition as a sales program within the university. Recommendations on recognition provided by Forbes et al. (2014) were similar, stating that buy-in from university and college administrations are requirements for a Center to be formed (p. 8). Finally, both organizations, as well as both the Forbes and Weilbaker articles recommended an advisory board (Forbes et al., 2014; SEF, 2019; USCA, 2020; Weilbaker, 2004). Forbes et al. (2014) provided more in-depth recommendations for a successful sales program based on a USCA member survey. For example, while USCA and SEF recommend external funding, the Forbes article proposed potential funding models and highlighted that the relationship between the sales center and the university fundraising department can impact the amount of funding that could be expected. Models summarized included various combinations of endowments, external fundraising by the sales center director, funds budgeted by the university, and other private funding. Center directors in the survey indicated that nearly all (97%) 39 participate in external fundraising activities, while 87% indicated that they were directly involved in 80% or more of all fundraising for the Center. The study acknowledged that donations were generally filtered through the university fundraising department or foundation and that for this reason, Center Directors need to develop a good working relationship with the Foundation (Forbes et al., 2014, p. 12). They suggested meeting with the foundation to explain how the Center will work with donors, and that this type of meeting should result in a better relationship with the foundation. Forbes et al. (2014) also supported several additional USCA requirements such as a Sales Advisory Board, a stipend for the Center Director for work beyond that of a typical faculty member, administrative help, and specific sales role play facilities and associated technology. A commonality among the research related to the content of a sales program was the need to align the curriculum with industry needs. While advances in technology may not be industryspecific, some industries may have requirements about selling techniques or customer knowledge. These industries may be regionally focused so it is important for a university to understand their marketplace, as well as have a realistic view of their abilities and limitations. Sales Program Curriculum Development Types of Courses Courses that should be included in a sales program have been reviewed historically, but older recommendations were of less relevance for this paper. Although useful to review for foundational course needs, these older articles were outdated due to the rapid technology changes that were occurring within sales. For this reason, publications from 2015 to the present were used to identify contemporary curriculum needs, other than specific instances for which older manuscripts added value. Two important sources in course identification were the USCA and 40 SEF websites. The mission of these organizations is to help further sales education and they are highly respected among universities committed to professional selling programs. Both organizations provide recommendations on the curriculum for new programs and both are widely used by marketing departments for guidance on a variety of sales program topics, including new or revised program design and course selection. Both organizations supply sound, research-based evidence and authority among university programs when envisioning the development of a sales program. USCA (2020) offers membership to universities that meet their specific criteria. USCA requires applicants for membership to first apply for associate membership by meeting 13 criteria. Among the associate member criteria set forth by USCA are the specifications for the required curriculum. To become an associate member, a candidate university must teach a minimum of two sales classes, with two levels of sales training (e.g., a personal selling and advanced selling course) with one being a prerequisite course for the other. To apply for full membership, a university must teach a minimum of three sales courses. SEF (2017) also provides recommendations for curriculum for universities interested in developing a professional sales program. Like the USCA, SEF curriculum recommendations for new programs are to include a minimum of three sales-specific courses. SEF provides guidance on the three courses, listed below. They also recommend at least one specialized course. Their guidance is for every program to offer the following: Introduction to Professional Selling Advanced Sales Sales Management 41 They encourage new programs to offer at least one specialized sales course. Specialized courses identified by SEF include the following: Financial Selling Key Account Management Sales Technology Sales and Social Media Supply Chain Sales Ethics in Selling Sales-specific internships are strongly encouraged by SEF. They further stated that a general education course such as public speaking and standard marketing courses such as Introduction to Marketing are not recognized as courses in the sales-specific curriculum. A systematic review of the literature related to sales curriculum was published in a comprehensive manuscript in the Journal of Marketing Education (Spiller et al., 2019). This publication, as well as others that are related to sales program curriculum design, were reviewed to ascertain current curriculum recommendations by authors from some of the most highly rated sales programs in the U. S. Some articles also identified courses that were predicted to be increasing in importance based on rapid technology changes. Spiller et al. (2019) provided results from a systematic review of sales education literature from 2014-2018, resulting in a comprehensive summary of findings from 53 articles. The publication also included curriculum information gleaned from a website review of all USCA member institutions, as well as results from a survey to university professors that included 57 SEF members and 30 USCA members and associate members. The website results 42 provided the most insight as to common sales program types and course lists. The program types identified are summarized in Table 3. Table 3 Common Sales Program Types at USCA Member Universities Sales Program Type Major and Minor Major only Minor only Certification Concentration/Specialization/Track Source: Spiller et al. (2019) % USCA Members (n=49) 22.4 28.6 42.9 26.5 24.5 Spiller et al. (2019) also reported that the majority of programs offer four to six courses in their sales program. The five most common courses among the 49 USCA members and associate members reviewed are the following: Professional Selling Advanced Selling Sales Management Negotiation Consumer Behavior Programs that consisted of a major or minor nearly always included an introductory sales course, an advanced sales course, and a sales management course (Spiller et al., 2019). A combination Selling and Sales Management course was more common in the 1980s, with nearly one-quarter of schools offering such a course, and the intent to offer such a course was growing among universities at that time (Kellerman & Hekmat, 1989). The combined course fell out of favor by 2000, when recommendations to split such a course into an introductory selling course 43 and a higher level sales management course were made by over 100 academics and business people at the AMA Faculty Consortium on Professional Selling and Sales Management (Marshall & Michaels, 2001). Spiller et al. (2019) also reported that additional specialty courses were offered depending on the total number of courses to meet requirements for the major or minor. Examples included some combination of Negotiations, Consumer Behavior, and Business-to-Business Management. Sales Analytics and Sales Technology were listed as relatively rare courses, but growing in popularity (Fogel, Hoffmeister, Rocco, & Strunk, 2012; Spiller et al., 2019). In addition, Spiller et al. reported that 34.7% of programs required a sales internship, and 44.9% offered online courses. While insightful, the Spiller et al. information had drawbacks. It summarized the state of USCA universities which were primarily large, well-established, and well-funded programs. These programs were not known for being nimble when it came to rapid curriculum changes to meet market needs. It also provided a view from late 2018, and with the rapid changes in digital sales technologies, may have overlooked new course opportunities that recently emerged. Therefore, this information was viewed as developmental, but not necessarily as distinctive or viewed as a requirement for a small university preparing to launch a new program. The lack of information on the accessibility of the program to students outside of a business school was also a drawback. A review of Stetson Universitys sales program provided a benchmark for a private university that implemented a sales program. Stetson launched a new sales program in the fall of 2018 and summarized their experience in a presentation at the 7th Sales Educators Academy in May of 2018 (Cox, 2018). Stetson has approximately 3,100 undergraduate students and is 44 located in Florida. They are an NCAA Division 1 School, and their business school is AACSB accredited. They offer both a major and a minor in Professional Sales, with the following requirements shown in Table 4. Table 4 Stetson Universitys Sales Major and Minor Course List Sales Major Six Courses SALS 230 Professional Selling and Communication SALS 330 Advanced Professional Selling and Communication SALS 410 Sales Management SALS 450 Executive Sales Strategy Plus 2 electives from a list of standard marketing courses Sales Minor Four Courses available to business and non-business majors SALS 230 Professional Selling and Communication MKTG 315 The Marketplace and Consumers SALS 330 Advanced Professional Selling and Communication SALS 410 Sales Management Stetson launched the sales program with both a major and a minor in just one year. They held an open house with 26 companies in attendance. Although financing was not addressed, they discussed the construction of five small role play rooms with video recording capabilities and one group role play room, as well as new faculty for the program. In their first year of launch, they reported that eight sections of the introductory sales course were offered, along with three sections of the advanced sales course. It did not appear that Stetson had applied for membership to the USCA (USCA, 2020, Membership List section). A second benchmark from the literature was provided by Newberry and Collins (2015), focused on Winona State University in Minnesota and their recent review and update of their sales program. The College of Business offers a minor in Professional Sales as part of their marketing program. The minor is open to business and non-business majors. Their sales program 45 is hosted in the Strauss Center for Sales Excellence. They are AACSB accredited, a USCA member, and are recognized as a SEF Top Sales University. Newberry and Collins reported that in 2015, the Center Director and one of the sales professors implemented a review of their existing sales program to identify critical instructional content and pedagogy. Their methodology included focus groups and in-depth interviews with sales practitioners, as well as a literature review. In addition to numerous content updates, a result of the study was to add a fourth course, a capstone experience, to their sales minor curriculum (Newberry & Collins, 2015). The resulting curriculum and course titles are shown in Table 5. Table 5 Winona State Universitys Sales Courses for a Minor Professional Sales Minor Requirements, 21 credit hours MKTG 265 Introduction to Professional Selling MKTG 300 Principles of Marketing (required as part of the Marketing major) MKTG 345 Sales Management MKTG Advanced Professional Selling Program Capstone (Independent Study on the Sales Competition Team or an Internship) Two additional electives from a list of marketing or communications courses Source: Newberry & Collins, 2015 Expected Knowledge and Skills and Typical Pedagogy in Sales Education Many publications outline the importance of technical and soft skills needed by marketing students (Bolander et al., 2014; Pink, 2012; Spiller et al., 2019). Sales research has shown that these are important skills to prepare students for success in professional selling positions (Healy, Taran, & Betts, 2011; Lilly & Stanley, 2016; Spiller et al., 2019). Sales education has been shown to produce these required outcomes (Jelinek, 2018; Lilly & Stanley, 2016; Loe & Inks, 2014; Spiller et al., 2019). Spiller et al. (2019) provided an inventory of skills, gathered from syllabi of 63 sales courses from SEF member professors. Their results showed that 46 sales educators heavily emphasized selling skills, communication skills, and relationship/ organizational skills, with over 70% of all syllabi reporting these course objectives. Other skills identified in the syllabi included networking, managerial, life, critical/analytical thinking, negotiation, problem solving, and technology. In addition to the educator perspective, it was important to understand the practitioner perspective when identifying skills needed for new graduates pursuing a professional selling career. The Newberry and Collins (2015) study provided a comprehensive view related to knowledge and skills in sales education from an industry standpoint. Using extensive background research, along with the qualitative research methods of focus groups and interviews of sales practitioners across different companies and industries, they identified the key selling skills that were preferred by sales practitioners. Their practitioner inventory of skills, categorized as critical, important, and useful, are shown in Table 6. Table 6 Sales Practitioner Skill Inventory Categorized as Critical, Important, and Useful Skill Inventory by Type Critical Communication Nonverbal: body language, facial expressions, grooming and fashion, proxemics Listening: focus, responsiveness, active listening Questioning: SPIN (Rackham, 1988), motivational interviewing Communication Styles: social style Writing Persuasion Self-Perception/Personal Expression: positive image, attitude, motivation, self-reliance, accountability, energetic, sociable, optimistic Professional Development: competitive, practice positive behaviors, entrepreneurial, coachable, organized, work efficiently, adaptable, demonstrate integrity Presentation: use of visuals, touching audience emotions, nonverbal, organization, providing a big finish, engaging the audience, voice clarity and use of tone/volume, being prepared. Important Networking: purpose, tools, processes 47 Rapport Building: meet and greet, use of questions and listening, communication styles Collaboration: teamwork, leadership, gaining by compromising, supporting others, with internal departments Critical Thinking: problem solving Negotiation: overcoming resistance, reaching a favorable commitment, conflict management Empathy Expression: focus on customer, use of communication skills, expressing understanding Time/Territory Management: goals and objectives, results oriented, establishing a plan, organization and scheduling, prioritizing customers/prospects, account planning Useful Research Techniques: leads, qualifying, identifying decision makers Closing Techniques Source: Reproduced from Newberry & Collins (2015) The previous studies supported that both sales educators and practitioners highly value communication skills such as listening, questioning, presentation, and persuasion or influence. Customer influence has further been documented in literature as including the ability to establish rapport, create a connection with buyers, uncover customer needs and wants through effective communication, presentation skills, and questioning to move the relationship forward (Bolander et al., 2014; Deeter-Schmelz & Kennedy, 2011; Marshall & Michaels, 2001; Moncrief & Marshall, 2005). Client entertainment activities also require skills related to communication and influence. Sales students, sales faculty, and sales practitioners rated entertainment activities, specifically those where alcohol is present, as important in achieving sales objectives and as a way to bond with co-workers, suggesting that managing these situations effectively was also a needed skill in sales education (Rodriguez, Honeycutt, & Ragland, 2015). Experiential teaching methods are well established in sales education, and research has established their effectiveness in achieving the desired learning outcomes (Inks & Avila, 2008). Spiller et al. (2019), in their review of prior sales literature, reported the most common teaching methods as lecture, role play, class discussions, in-class exercises, student presentations, and 48 projects, with 75% or more reporting those methods. Fifty to 74% reported assessments, group/team activities, and guest speakers. Additional teaching methods were reported less frequently overall but at a higher frequency in specific courses, such as videotaping, competitions, and coaching (Spiller et al., 2019). Deeter-Schmelz and Kennedy (2011) also reported that from a survey of 47 universities offering sales programs, 97.4% of universities used role play, 97.4% lecture, 86.8% guest speakers, 85.9% videotaping, and 84.2% in-class exercises. The USCA (2020) lent credence to the importance of role play, suggesting the use of mock selling situations and reporting that the top selling schools have established role play rooms where students are able to apply influence tactics in videotaped sessions. Newberry and Collins (2015) reported on pedagogy from the standpoint of sales practitioners. Their practitioner inventory of pedagogy, categorized by critical, important, and useful, is shown in Table 7. Table 7 Sales Practitioner Pedagogy Inventory Categorized as Critical, Important, and Useful Pedagogy Inventory by Type (*denotes an experiential pedagogy) Critical Role Play: competitive, own creation, given scenarios, computer generated* Actual Sales Experience: internships, fundraising sales calls, class projects, mentors* Job Shadow/Observation* Extra-Curricular: competitive sales team, networking events, seminars/workshops* Important Speakers: live, online Useful Simulations: online, computerized* Lecture: traditional, interactive Exercises/Assignments Exams/Quizzes Research/Independent Study Source: Newberry & Collins (2015) 49 Sales Technology and Ethics in Professional Selling Programs There appeared to be agreement on three courses for a sales curriculum (e.g. professional sales, advanced professional sales, sales management), general acceptance of at least one more (e.g. negotiation), and an acknowledgment among sales program leaders that the rapid changes emerging as a result of technology would create a need for inclusion of sales technology-oriented content. Topics such as sales analytics, social media in sales, and technologies such as sales force automation (SFA), customer relationship management (CRM), and artificial intelligence (AI) are part of the new sales technologies in use in corporations. Several of the larger USCA member universities with a major in sales offer some type of sales technology content, but this type of course was uncommon among universities offering only a minor or specialization (Spiller et al., 2019). Jelinek (2018) explained that the lack of technology in sales programs was because sales technology is inherently fast-changing, (p. 80) and instructors face a significant challenge when it comes to handling SFA in the classroom (p. 80). This challenge is predicted to increase with the emergence of the latest sales digitalization technologies like chatbots, social media, mobile apps, and AI, which are likely to be more significant and more far-reaching than previous sales technologies (Kunkle & Peterson, 2018; Singh et al., 2019). In a team-based workshop and collaborative process at the Setting the Research Agenda in Sales session at the 2018 AMA New Horizons Faculty Consortium in Selling and Sales Management, teams concluded that digitization and AI technologies pose a disruptive force and that AI technologies were triggering fundamental changes to the value creation function in sales (Singh et al., 2019, p. 2). 50 This was explained in part through a concept called sales enablement. Singh et al. (2019), reports that: Sales enablement groups are being established in many firms with a focus on providing salespeople with technology and information to sell more efficiently. These sales enablement roles are primarily responsible for drawing insights from sales activity data and creating digital content for sales leaders and salespeople to improve performance (p. 10). These technological changes also impact the idea that sales professionals may become knowledge brokers and improvisational thinkers who must integrate intellectual capital from digital technologies with their knowledge of the selling process (Kunkle & Peterson, 2018; Singh et al., 2019). Kunkle and Peterson (2018) also surmised that skills such as empathy, listening, critical thinking, and problem solving, often displayed by elite B2B sales professionals, would increase in value when combined with these new sales technologies. Due to the advances in technology, especially related to AI, and the resultant customer information that is resident in these systems, sales ethics is also a growing concern (Singh, 2019). Few universities offer specialized content in this area, yet student perceptions of selling ethics have been known to be a primary predictor of intent to pursue a selling career (Peltier et al., 2014). A review of USCA programs offering the Bachelor of Science in professional selling revealed no instances of a standalone sales ethics course (USCA, 2020). Others reported that among AACSB schools, 19 25% required a business ethics course of all business students (Bowe, 2015). 51 There has been a long-standing gap in ethics content within sales education, yet salespeople, due to their cross-functional interactions within and outside the company, are highly likely to confront complex ethical dilemmas (Anderson, Dixon, Jones, Johnston, LaForge, Marshall, & Tanner Jr., 2005; Bush et al., 2014). As early as 2004, the American Marketing Association called for increasing sales ethics content and application within the classroom (AMA Summer Conference, 2004). Most sales educators agreed that any sales program should include some significant focus on sales ethics, yet how that content was to be incorporated at the program level was not well explored. What educators did appear to agree on was that ethical content should include a variety of experiential tools that allow opportunities for students to analyze, apply, and discuss ethics and the importance of ethical decision making (Anderson, et al., 2005; Bowe, 2015; Castleberry, 2014; Cummins, Peltier, Erffmeyer, & Whalen, 2013; Cummins, Peltier, Pomirleanu, Cross, & Simon, 2015; Donoho & Heinze, 2011). There was also evidence in the literature that students who completed courses in ethics showed stronger ethical decision making skills and more positive perceptions of salespeople, yet also that these results could be achieved through business ethics courses aimed at all business majors, and not necessarily sales ethics specific courses (Bowe, 2015). It is unknown if this conclusion will continue to hold under the rapidly changing landscape of technology, namely AI and machine learning (ML). Singh et al. (2019) recommended research on the ethical implications of AI-based value creation and sales communication and as this research becomes available, it may highlight additional sales-specific ethics training needs. What did appear to be clear was that sales technology and sales ethics were two areas of potential interest in new program development, and as the two continue to become more intertwined through the emergence of AI and ML technologies applied to the sales function, sales programs will have to adapt. 52 Evaluation of Sales Education Programs within Business Schools Accrediting Agencies for evaluation Sales Program in Higher Education Schools offering a sales major, e.g. a degree in professional selling, may be subject to evaluation of that degree from external business accreditation agencies. There are three Councils for Higher Education Accreditation (CHEA) recognized business school or business programmatic accreditation agencies in the U.S. used by business schools. Common membership profiles are summarized in Table 8. Table 8 Summary of Business School Accreditation Agencies Agency Description AACSB International Association to Advance Collegiate Schools of Business International. Very high presence among research oriented mid to large sized State universities. ACBSP Accreditation Council for Business Schools and Programs. Most members are mid-sized teaching-oriented institutions. Significant representation of for-profit private colleges (Most members are small teaching focused private liberal arts colleges. For-profit colleges, thanks to increased scrutiny over their practices, seem to consider ACBSP accreditation as critical to validating their quality). IACBE International Assembly for Collegiate Business Education. Most members are small teaching-focused private liberal arts colleges. Source: George (2018) The majority of sales programs reside in business schools holding AACSB accreditation (SEF Annual, 2020). AACSB (2020) has specific evaluation standards for any major leading to a business degree, such as a Bachelor of Science in Professional Selling. The other two accreditation agencies utilize similar standards for program (major) evaluation. AACSB standards classify a minor, concentration, specialization, certificate, or track as a microcredential. They expect that these types of credentials should be reviewed for quality; however, a formal Assurance of Learning (AoL) systematic process with competency goals in 53 each program is not required. The other two accrediting agencies have similar requirements in that only programs resulting in degrees (e.g. majors) are formally evaluated, not minors or specializations. For this reason, programs that offer anything other than a major in Professional Selling are not required to publicly report accreditation results related to their sales programs (AACSB, 2020). The AACSB accreditation process can be a useful tool for the development of an educational evaluation framework. Lakhal and Sevigny (2015) summarized the AACSB AoL process as these five steps: 1. Define student learning goals and objectives 2. Align curricula with the adopted goals 3. Identify instruments and measures to assess learning 4. Collect, analyze, and disseminate information 5. Use information for continuous improvement Step 3, the identification of instruments and measures to assess learning can be accomplished using any of three acceptable approaches under AACSB standards. These are selection, course-embedded measures, and demonstration through stand-alone testing or performance. Selection is accomplished if students are selected to participate in the program if the learning goals have already been developed and met through their prior academic experiences. Course-embedded assessment is the process of using artifacts generated through classroom activities to assess achievement of student learning objectives at a program level. It builds on the daily work (assignments, tests, projects, etc.) of students and faculty members, (McConnell, Hoover, & Miller, 2008. p. 20). The option of demonstration occurs at the end of 54 the academic program and utilizes assessment tasks outside the normal classroom that are generally designed by the faculty, according to Lakhal and Sevigny (2015). In practice, however, they reported that demonstration is frequently measured in the classroom unless it is being accomplished by a standardized exam, such as a graduation exam. No matter the choice, they concluded that a benchmark must be determined to evaluate acceptable performance. Other Examples of Sales Program Evaluation in Higher Education Aside from formal accreditation requirements for programs with sales majors, academic literature regarding the evaluation of sales education programs based on outcomes was very limited. Bolander et al. (2014) offered the following statement when writing about the lack of outcomes studies in this area: Nevertheless, one subfield of marketing, professional selling, has for years proffered anecdotal claims that performance of graduates with formal sales education is higher, and turnover much lower, than their peers (Fogel, Hoffmeister, Rocco, & Strunk, 2012). However, a search of the scholarly, peer-reviewed literature reveals no empirical validation of these claims. One is left to wonder to what extent these assertions hold beyond the wishful thinking of the sales educators that make them! (p. 169) Kirkpatricks four-level model (1996) has been applied in non-academic, organizational settings as a method to evaluate sales training programs, notably in the retail industry (Tan & Newman, 2013). Praslova (2010) proposed the use of Kirkpatricks model in academic program evaluation, suggesting it served as a comprehensive and systematic approach to aligning criteria for educational effectiveness with specific indicators of achievement of these criteria... (p. 215). Blooms taxonomy (1956) has been utilized at the course level as a means for the evaluation of skills within sales courses. Healy et al. (2011), in the Journal of Instructional 55 Pedagogies, utilized Blooms taxonomy and experiential learning theory to examine two courses in a successful professional sales program. Experiential learning is rooted in hands-on practical exercises and experiences, while Blooms taxonomy was used by grouping objectives into three groups: low level knowledge or understanding; medium level application or analysis; and high level synthesis or evaluation. They combined the two frameworks to show how experiential learning in the sales classroom could incrementally move students to higher levels of learning. They used the professional selling and negotiation courses as examples. This combined framework, shown in Table 9, provided specific components mapped to the phases and modes of experiential learning and the levels of Blooms taxonomy. Table 9 Mapping Course Components for Experiential Learning and Blooms Taxonomy Grasping Experience Transforming Experience Concrete Experience Abstract Conceptualization Reflective Observation Active Experimentation Knowledge and Understanding Past Experience Course Readings, Lectures Homework, Quizzes n/a Application and Analysis In-Class Exercises, Early Role Plays Class Discussions, Early Role Play Debriefings Role Play Write-Up In-Class Exercises, Early Role Plays Advanced Role Later Role Plays, Later Role Play Play Write-Up, Sales Challenge Debriefings, Sales Sales Challenge Participation Challenge Feedback Term Paper Source: Reproduced from Healy, Taran, and Betts, 2011 Synthesis and Evaluation Later Role Plays, Sales Challenge Participation At the assignment level within a single course, Mullen and Larson (2016) reported on a set of learning outcomes related to a sales shadowing program at Georgia Southern University. Seven learning outcomes were identified based on Blooms taxonomy and students were 56 evaluated through the use of rubrics. Although the outcomes were developed specifically for this job shadowing exercise, they provided an example of how Blooms taxonomy could be used in the development of an evaluation framework. The seven learning outcomes for the sales shadowing assignment were as follows: 1. Student will be able to prospect, select, prepare, and plan a sales call 2. Student will demonstrate evidence of completed research and successful application of planning a sales call 3. Student will apply business communication skills by constructing a letter to the sales manager 4. This assignment simply ensures that students are on schedule, and can coincide with a time management discussion 5. Student will critique and reflect on the sales meeting. Student will apply concepts and skills taught in classroom to observations from a real-world experience 6. This assignment supports discussion on professionalism, relationship management and follow-up, and can coincide with a CRM discussion 7. Student will be able to receive constructive criticism on their performance outside of the classroom instructor. The evaluation provides feedback from the managers perspective on the interaction, and also asks for the managers interest in future shadowing projects Mullen and Larson (2016) also reported percent job placement and percentage of internship offers among participating students as evaluation measures for the exercise. An increase in support from the companies involved, in terms of time commitment, financial support, and expressions of desire to strengthen the partnership with the university were reported as additional positive measures resulting from the job shadowing (Mullen & Larson, 2016). A methodology for the development of program outcomes was suggested by Bolander et al. (2014). They utilized survey methodology and statistical analysis to assess the relationship between what is learned in university sales programs and the actual selling behaviors of recent 57 program graduates. The methodology used allowed a comparison between students who did and those who did not receive formal sales education in their undergraduate programs. The results supported the relationship between formal sales education and sales performance and the authors concluded that salespeople hired from universities with a formal sales program outperform their non-sales program counterparts after the first year of employment (pp. 176-177). Wood (2020) also used a survey to evaluate the sales career success of graduates of a sales leadership course. Both the students who had completed the course and their supervisors were surveyed and results showed that the sales leadership course resulted in increased leadership behaviors demonstrated post-graduation. Surveys have also been used to evaluate course pedagogy such as role plays and video recording, concluding that these techniques for sales education were perceived as effective or more effective than other teaching methods (Deeter-Schmaltz, and Norman Kennedy, 2011). Beyond these few publications and examples, established outcomes, measurements, assessment instruments, or benchmarks at a sales program level are very limited in the existing literature. Specifically, research related to an overall evaluation framework for a sales program within higher education, and the criteria for measuring student outcomes throughout such a program, was lacking, creating a significant gap in the literature. 58 Chapter 3: Methodology This project used a mixed methods methodology. Mixed methods research is a type of research in which a researcher or team of researchers combines elements of qualitative and quantitative research approaches (e.g., use of qualitative and quantitative viewpoints, data collection, analysis, inference techniques) for the broad purposes of breadth and depth of understanding and corroboration (Johnson, Onwuegbuzie, & Turner, 2007, p. 123). Mixed methods research provided several advantages for this study. For example, qualitative research was used to explain, clarify, and provide meaning to the information supplied in a literature review. Quantitative research provided student perspectives that were useful in supplementing the qualitative interviews and confirming the curriculum framework developed as a result of those interviews. Mixed methods was also useful since there were different stakeholders needed for data collection, all were small populations, and there were differences in specific areas of expertise for each group. Mixed methods research allowed for data collection from each group while adding to the credibility and validity of the findings and limiting bias related to a single methodology. For these reasons, mixed methods research allowed for a greater depth of knowledge from each group to inform the resulting recommendations for program development. No ethical concerns were identified with this methodology. Potential bias existed in the interpretation of the data by the researcher, who also created the program being evaluated. The methodology outlined in the study was developed to assist in mitigating this issue. The use of standardized rubrics completed by multiple different faculty or external sales experts, and objective evaluation data collected independently from sources other than the researcher are part of the methodology that addressed bias concerns. All data were held securely per the universitys institutional research policy and any identifying information, if collected, was eliminated from 59 the data files before they were stored on any electronic device or media. The methodology included four parts: Part 1: Benchmarking best practice universities Part 2: Qualitative inquiry for program development Part 3: Quantitative survey of program students Part 4: Program evaluation o Level 1: Reaction o Level 2: Learning o Level 3: Behavior o Level 4: Results The analysis of Parts 1 - 3 was used to create the program design. It included the identification of the knowledge and skills as outlined in the research question, as well as verification of the courses that were needed to deliver the identified knowledge and skills. Part 4 consisted of the program evaluation and was based on the adapted version of Kirkpatricks four level model (Kirkpatrick, 1996, 2005; Kurt, 2016; Praslova, 2010). Methodology Part 1: Benchmarking best practice universities Five institutions were benchmarked using secondary information, primarily university and other public websites. Programs selected for benchmarking included two regional, medium to large sized state universities, two small to medium Midwestern Catholic liberal arts universities, and one aspirational university. The aspirational university is a medium to large sized private university with a well-respected, successful professional selling program. The universities are labeled in this study as Regional, R1 and R2, Catholic, C1 and C2, and 60 Aspirational, A1. Benchmarking data supplemented information found in the literature review. The benchmarking process also informed interview questions in the interview guide development process for Part 2: Qualitative inquiry for program development. Methodology Part 2: Qualitative inquiry for program development Six qualitative interviews assisted in defining the knowledge and skills needed by sales program graduates. Two sales program directors at other schools were interviewed to gain perspectives on how other sales programs were developed, what they teach, and how they assess their programs. Three interviews were with current business school partners. These interviews allowed for a deeper understanding of the needs of local businesses regarding the desired student preparation prior to hiring. The final interview was performed with a senior member of a nonprofit organization that supports research and the promotion of university sales programs. This interview was used to gather a broader and research-based perspective on knowledge, skills, and coursework. The six interviews were implemented using in-depth semi-structured interviews. The interview guide used a logical flow from general to more specific questions that were discussed in a consistent manner, with probes designed to elicit more elaborate responses. This type of interview is flexible, accessible, and considered to be ideal for qualitative interviews (Adhabi & Anozie, 2017). The semi-structured interview guide was prepared and conducted by the researcher, who has been trained as a professional in qualitative research applications and techniques such as the following: interview guide development moderator training through the Burke Institute (Burke, 2020) part of coursework in the researchers doctoral program 61 The research instruments are included in Appendix B. Interview participants were sent an email invitation informing them of the purpose of the email, the purpose of the request for an interview, and the request for their voluntary participation. Five interviews were held in-person. For the remaining interview, the invitation to participate specified a telephone appointment to be scheduled. The appointment was scheduled for one hour and it lasted approximately 45 minutes. There was no compensation for the interviews. Interview sessions were not recorded. The interview guide was not sent to the participants before or after the interview. The six interviews were held with the following groups. Faculty at Other Schools. Two sales program directors were interviewed to gain insight from their experiences running successful sales programs at their universities. Both of their universities were USCA members. The interview topics included key learnings from their experience, sales program management, partnership sources and management, curriculum, pedagogy, knowledge and skills, and future plans. Those selected in this sample were chosen based on convenience, proximity, and willingness to participate. Business Partners. Three sales executives previously involved in the sales program were interviewed. Interviews focused on partnership needs and wants, partnership management, experience with other universities, and knowledge and skills of new graduates. Two interviews were conducted with executives at Fortune 1000 insurance and investment firms. A managing director was interviewed during the spring of 2020 and a Director of Internship Recruitment from a large regional office was interviewed in February of 2021. The third interview was held in the spring of 2020 with a District Manager at a nationally recognized logistics and distribution company. Both companies involved in the interviews were actively promoting internships and full time positions to sales students in the business school at the time of the interviews. 62 Non-Profit Organization Supporting University Sales Programs. One interview was held with a senior member of this organization. This interview focused on starting and growing a sales program, key learnings from other universities, curriculum and pedagogy, student knowledge and skills development, the value of their organizations to universities, how small universities compete and grow, sourcing and managing partnerships, and upcoming changes and advances that may impact sales programs. Methodology Part 3: Quantitative survey of program students A quantitative survey was administered to current sales students in the business school. The survey instrument consisted of 40 data points, and a copy is included in Appendix B. The sales program benefits section of the survey consisted of one question with six data points. This question was developed by the researcher due to its institution-specific content. Another section of the survey used the Intent to Pursue a Sales Career (ITPSC) 13 item short-form scale (Peltier et al., 2014). The ITPSC instrument used for this study was approved by the ITPSC author and copyright holder (Peltier et al., 2014). An additional survey section related to learning specific skills within courses taken in the business school was designed by the researcher and used a standard five-point Likert-type scale from strongly agree to strongly disagree. For this section, question wording was informed from previous sales education surveys identified in the literature (Barat & Spillan, 2009; Bolander, et al., 2014; Bristow et al., 2006; Peltier et al., 2014). The survey was fielded using the Qualtrics survey tool in the online format. Survey invitations were sent using the university email addresses of all members of the relevant population of current students who had taken or were taking a sales class during the spring 2021 or fall 2021 semester. All invitations informed the potential respondent of their rights, estimated survey completion 63 time, and individual respondent confidentiality. All other Institutional Research Board procedures were followed. No incentives were offered. The goal of the student survey was to assess current perceptions of the knowledge and skills gained through sales courses that had been recently implemented at the university in this study, and their alignment to the knowledge and skill needs that were identified in Part 1: Benchmarking best practice universities, and Part 2: Qualitative inquiry for program development. Survey results also helped confirm that the overall program objectives were being met, and assisted in the identification of any needed adjustments within individual courses. Additional survey questions were used to assist in future program development and were limited in their applicability to the research question under study. These additional questions centered on perceptions of sales careers and the identification of needs and wants relative to course topics, skills, competitions, facilities, external partnerships, and future career aspirations. Sample Size. The population surveyed was a census sample of current university sales course students, with the sample list drawn from all sales course rosters. The population was 109 students, comprised of students from the spring 2021 and fall 2021 semesters. Email invitations were sent during the final four weeks of the semester for spring and fall 2021. Participants were given two weeks to respond, with a reminder email sent to all non-responders one week after the initial invitation. The total number of usable survey responses was 63, resulting in a 57.8% response rate, a good representation of the participants. Methodology Part 4: Program evaluation The program evaluation model that was implemented was based on Kirkpatricks four level model (Kirkpatrick, 1996, 2005; Kurt, 2016). This model was chosen based on its 64 traditional use in business and organizational contexts, as well as its adaptation specific to higher education program evaluation (Praslova 2010). Arthur, Tubre, Paul, & Edens, (2003) utilized Level 1 and 2 of the four levels model for evaluation of teaching effectiveness in higher education. Level 1: Reaction, and Level 2: Learning, share similar characteristics across all training and educational settings and were therefore easily applicable to higher education. Applying Level 3: Behavior, and Level 4: Results, to higher education required some adaptation of the model. Praslova (2010) proposed this adaptation specific to the context and purposes of the university setting. In this adapted model, the four levels remained the same, Reaction, Learning, Behavior, and Results, but the evaluation criteria and suggested evaluation methods differed for Level 3 and Level 4. Table 10 shows the adapted model used for this program evaluation. Table 10 Sales Program Evaluation Model using the Adapted Kirkpatrick Four Levels Criteria Level 1: Reaction Level 2: Learning Level 3: Behavior Learning in Higher Educationa Student affective reactions and utility judgments Direct measures of learning outcomes, knowledge tests, performance tasks or other graded work Evidence of student use of knowledge and skills learned in the program and applied in subsequent work, e.g., internship, development of a professional resume, and other behaviors outside the context in which initial learning occurred Alumni career success, graduate school admission, service to society, personal stability a Taken from Praslova (2010). Level 4: Results b Indicators and Instruments Student evaluations of instruction all sales courses included Pre and post-evaluation of knowledge and skills in the classroom setting in the introductory and advanced sales courses End-of-course final project in 400-level sales coursesb, supervisor internship evaluation of all students participating For all sales program students: Job Placement Rate for Graduates, Starting Salaries, Job Titles Advanced Sales course competition results; Negotiation final project. 65 For this project, the Level 2 and 3 results were used to measure sales competencies and answer the research question being studied. Level 1 and 4 results did not explicitly address the research question. They were identified as necessary for a comprehensive review of the new program being implemented and are reported for that purpose. Sampling The sample for Part 4: Program Evaluation, Levels 1 through 4, was a census sample of all students participating in the sales program between May 2020 and May 2022. Research Instruments and Data Collection For Level 1: Reaction, the student evaluation of instruction questions are included in Appendix C. Data were collected as part of the university course evaluation process. All students were invited to participate in the student evaluation of instruction for each course. For Level 2: Learning, pre- and post-evaluation of knowledge and skills was assessed using a rubric for signature role play assignments in the introductory sales class and in the advanced sales class. These rubrics are included in Appendix C. Data collection was performed using direct observation by the course instructors during the first five weeks of class and then again during the final four weeks of class to provide the opportunity to evaluate growth during the semester (within group) as well as growth between the introductory and advanced course levels. All students in the courses were included. Level 3: Behavior, included three types of measures. First, end-of-course final projects in two senior-level courses were used. The advanced selling course used an external sales competition rubric and result as the measure of knowledge and skills applied. The negotiation course used the final negotiation role play scores. Rubrics used are included in Appendix C. The next measure in Level 3 was an internship evaluation done by supervisors of student interns. For 66 each sales student who had completed a sales internship, the results of the supervisor evaluation for the two skills relevant to the sales program were collected and recorded. These data are collected by the universitys career services office as part of the internship experience. The rubric used for this evaluation is included in Appendix C. For Level 4: Results, data collected included job placement rate, starting salary, and job title for each sales program graduate. These data were collected by the career services office. Data were summarized in an Excel spreadsheet as follows: Job placement rate of sales program graduates reported as a percentage of students obtaining a full time sales position within six months of graduation Average starting salary of sales program graduates, reported as a mean value and a median value of student-reported salary, collected from the university survey of new graduates within six months of graduation Job titles of sales program graduates who obtained a full time sales position within six months of graduation in a list format Data Analysis Plan Part 1: Benchmarking best practice universities The benchmarking results were reviewed for similarities and differences. Findings were used to inform other parts of the methodology for the project. The data collected were summarized in tabular form. Part 2: Qualitative inquiry for program development Qualitative data from the interviews were analyzed in three steps: data reduction, data display, and conclusions and verification (NSF, 1997). 67 In data reduction, written notes from the interviews were transcribed. Common patterns and themes related to knowledge and skills were identified and categorized. The knowledge and skill categories and themes were then grouped according to common terminology and displayed in a matrix to identify patterns and critical points of agreement and difference to understand the needs of each stakeholder type. Verification of the findings from this step consisted of revisiting the original interview notes to cross-check and verify the emergent conclusions. The display matrix was assessed for meaning and used in the final program design. Part 3: Quantitative survey of program Survey analysis was completed after downloading all survey results into a Microsoft Excel spreadsheet. The analysis consisted of frequencies, cross-tabulations, measures of central tendency, and measures of variance. Statistical treatments, such as a t-Test of means, were implemented where relevant. Open-ended comments were not relevant and therefore not utilized in this analysis. All results from Parts 1 through 3 assisted in creating the professional selling program design and the development of the needed knowledge and skills to be included in each course. Part 4: Program evaluation The data analysis plan for the sales program evaluation was implemented for each of the four levels in the Kirkpatrick model. Level 1: Reaction. Data were provided through the universitys course evaluation software. These evaluations consisted of seven questions related to the course and six questions related to the course instructor. Each question was evaluated by the student using a five-point verbal scale of strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree. Results were analyzed by reviewing the means and standard deviations shown by question for 68 each sales course offered. Differences between semesters, where available, and between introductory and advanced course levels were compared. Mean scores were used to assess student affective reaction and utility judgments on the seven criteria rated for each course. Level 2: Learning. This evaluation utilized a pretest-posttest design for the introductory level and the advanced level sales courses. The nine core selling skills were evaluated using role play scenarios and a rubric that measured five sales concepts during the role play. The rubric included levels of evaluation ranging from Mastery to Not used at all. The maximum points (e.g. Mastery) for each category was 10 and the maximum point for the overall score was 50, or the sum of the five category points earned. The analyses included the following: descriptive statistics, including frequencies, for each sales concept percent of students achieving the mastery level within group (within a class section by level) comparisons, pre- and post-intervention, using a paired sample t-Test where pre and post level was nominal and total points earned on the rubric was interval level between group comparison of end of semester scores (introductory versus advanced sales course), using an independent samples t-Test Level 3: Behavior. This evaluation consisted of two different analyses including end-of- program projects and internship performance. The end-of-program final project data were gathered from rubrics for evaluating the final projects in the two 400-level courses offered. Each student was evaluated by a mix of sales professionals and course professors. The analysis consisted of the following: Descriptive statistics, including frequencies, for each knowledge or skill assessed Percent of students achieving the mastery level 69 Identification of areas of strength and areas with opportunities for improvement among the course participants overall Each student completing a sales internship was evaluated by their supervisor on communication skills and teamwork. Mean scores were reported along with a comparison to all business school student internship scores. Level 4: Results. This evaluation consisted of obtaining and reporting on the following metrics, along with comparisons to business students overall and the industry averages from the National Association of Colleges and Employers (NACE) First Destination Dashboard (2019): Job placement rate, defined as obtaining full time, professional placement in a business role within six months of graduation, as reported by the universitys career center Starting salary, self-reported by the graduate in the student exit survey or as reported by the universitys career center Job title, as self-reported by the graduate in the student exit survey or as reported by the universitys career center 70 Chapter 4: Study Results To prepare students for success in professional selling careers, universities need to deliver a curriculum that teaches the desired knowledge and skills. This study was designed to examine the curriculum needs when creating a new selling program. The goals of this study were to a) identify the knowledge and skills needed by a graduating student pursing a sales career, b) implement the findings in current or newly formed courses that form a professional selling program of study, and c) evaluate the effectiveness of the program. A four-part, mixed methods study design was utilized. First, secondary research was performed to benchmark five successful programs in Part 1. This was followed by Part 2, a qualitative step using in-depth interviews to gather insights and perceptions from academic experts heading sales programs, business executives who hire and train new graduates in sales roles, and an industry expert from an organization focused on university sales education research. Parts 1 and 2 were used for the initial program design. Part 3 consisted of two occurrences of a quantitative survey of sales students within the program under study. Part 3 of the study was used to confirm the initial program design, including the knowledge and skills to be introduced and reinforced in each course. Part 4 of the study was the evaluation. The study was guided by the research question: What knowledge and skills do students need in preparation for a professional selling career, and how does implementing these in a formal curriculum improve students' sales competencies? Sales Program Design and Implementation, Parts 1 through 3 Program Design Part 1 Results: Benchmarking Similar, Regional, and Aspirational Programs Five university sales programs were selected for benchmarking. A review of the university and program websites were used to document the main characteristics of each 71 program, the knowledge and skills taught in the introductory and advanced sales courses at each school, and the types of sales courses offered. Two schools were selected based on their characteristics of Catholic, small to medium in size, and private liberal arts focus. They are labeled C1 and C2 in the following tables. Two regional schools were selected based on proximity to the university being studied in this project and are labeled R1 and R2 in the following tables. One school was selected as an aspirational school and is labeled A in the following tables. The aspirational program is well known as a national leader in undergraduate sales education. All schools chosen are members of the USCA. The main characteristics of each program are shown in Table 11, including an indication of which schools offer a sales major, minor, certificate, or specialization as part of their program. Table 11 Sales Program Characteristics School C1 C2 R1 R2 A Years program has existed 23 5 25 31 35 Sales Accreditation Students 40 70 250 150 452 ACBSP ACBSP AACSB AACSB AACSB Major Minor Certificate Specialization Table 12 documents the stated knowledge and skill outcomes for the introductory level sales course and the advanced level sales course at each benchmarked university. Two courses at each university were identified for this comparison based on the USCA guidelines stating that member programs must offer an introductory and an advanced sales course, and one must be a 72 pre-requisite for the other. Table 12 indicates when the knowledge or skill was specifically stated as a learning outcome for the course. Table 12 Knowledge and Skills Outcomes at Comparison Schools Knowledge/Skill C1 C2 Buying Motives A IA Persuasive Communication IA IA Sales Process IA IA Professionalism I Sales Ethics A Presentation Skills A A Teamwork A Time Management A Career Management I I Negotiation A I Salesforce Management A Relationship Management A IA Sales Technologies A A Note. I = Introductory Course A=Advanced Course. R1 I IA I I I IA I IA A A R2 I I I A A A I A A IA IA I I A A I A Table 13 summarizes the course types offered in each benchmarked program. Only courses that included the word sales or negotiation in the course title were included in this comparison. Course types were identified through both course titles and descriptions. Specific titles varied slightly by the university. The number of sales courses ranged from three to six. In addition to the introductory and advanced course, three schools also offered a sales management course and three offered a negotiation course. 73 Table 13 Sales Course Types Offered by Program Course Type Introduction to Selling/Principles of Selling Advanced Selling Sales Management/Salesforce Management Negotiation Sales Internship Professional Sales Practices Sales Ethics Specialty Sales (healthcare) Sales Technologies Sales Strategy TOTAL OFFERING C1 6 Comparison School C2 R1 R2 A 5 5 3 5 Sales Program Design Part 2 Results: Qualitative Inquiry Six in-depth qualitative interviews were completed to gain additional insight into sales program design and curriculum development. Coding was performed manually using the transcribed notes from all interviews. Additional documentation was provided by two of those interviewed. The curriculum, knowledge, skills, and pedagogy recommendations summarized from the interview notes can be found in Appendix D. Descriptions of knowledge and skills were classified using common terminology. For example, phrases or words describing body language, attire, expression, and use of professional language were grouped into a professionalism knowledge area. Meaning was extracted through conceptually meaningful words and phrases and then four themes were derived through a process of refinement and revision. These four identified themes were: 1. Curriculum Development; 74 2. Knowledge and Skills needed in sales education; 3. Pedagogy; 4. Funding of Sales Programs. Three of these themes were identified as relevant to this study. One theme, funding of sales programs, was less relevant to the research question being studied, and therefore was not analyzed further. Sales Program Curriculum Development and Implementation Results from Part 1, benchmarking, and Part 2, qualitative inquiry, were evaluated to define the sales program. This definition included three overall program learning outcomes, courses to be offered, and learning objectives for each course. The evaluation included assimilating all Parts 1 and 2 findings to determine a course framework. This framework specified nine core knowledge and skill attributes needed in the introductory course. These are: listening, questioning, the sales process, building trust, persuasion, communication, professionalism, building rapport, and solving problems. Ten additional attributes were identified for inclusion in the advanced sales courses, with a natural progression of introduction, reinforcement, and assessment as students move from the introductory course to the more advanced courses. Suggested pedagogies were identified to support the progression of fundamentals in the introductory course and more advanced selling skills and attributes in the advanced sales courses. The course framework can be found in Appendix D. The proposed program was then compared to the Part 1 and Part 2 detailed notes, documents, and findings to ensure consistency. Finally, an in-person review of the program was held with the two faculty members teaching courses in the sales program to confirm that all information was understood and accepted for implementation. This ensured that all course 75 revisions and new course objectives would be met during the next course offering. After agreement among the sales faculty, a review and approval was completed with the remaining business faculty, the dean, and the assistant dean. The program was then submitted for the formal university approval process. A nine-credit hour specialization in professional selling was approved by the university in early 2020. The three overall program learning outcomes were that students who complete the specialization would: 1. Understand the sales process. 2. Understand sales concepts and how to apply them. 3. Know how to structure a sales presentation. The approved program required the completion of the introductory sales course, which was already being offered once per academic year, an advanced sales course, and one additional course from a list of options. The additional course option included a negotiation course offered for the first time in spring of 2020, and a sales technology course that was offered for the first time in the fall of 2021. Students could also elect to complete a sales-specific internship for three credit hours. Sales Program Design Part 3 Results: Quantitative Survey of Students To confirm the sales program framework that was implemented, a quantitative survey was performed with students completing and of the courses. The survey was fielded at two different time periods, April 2021, November 2021. This allowed for a representation of students completing all courses during fall 2020, spring 2021 and fall 2021 semesters. Sample size and response rates for the survey are shown in Table 14 and survey demographics are shown in Table 15. 76 Table 14 Sampling Data from Student Survey Survey Date Apr 2021 Sample N 62 Completed Responses n % 34 54.84% Nov 2021 47 29 61.70% Total 109 63 57.80% Table 15 Survey Participant Demographics Characteristic Gender Female Male Prefer not to say Year in school Graduate in 2021 Graduate 2022 or after Note. n = 63. n % 23 39 1 36.51 61.90 1.59 15 48 23.80 76.20 Survey results are presented in the following tables using totals from both survey occurrences. Some students completed the survey after taking multiple sales courses. For this reason, the introductory sales course analysis included only those respondents who had not yet completed any other sales course. The advanced sales course analysis included students who had completed, at minimum, both the introductory and the advanced sales courses. This allowed for a comparison of knowledge and skills obtained in the first sales course, as compared to the growth in knowledge and skills from completion of the advanced sales course. For the other two courses shown, students may have completed any combination of courses. Survey results show that eight of the 63 respondents, or 12.7%, completed more than two courses at the time of the survey. 77 Students selected all of the courses they had completed previously or were completing during the survey period. Table 16 shows these results. Table 16 Sales Courses Completed Students Course Introductory Sales Advanced Sales Negotiation Sales Technology n 49 18 22 17 % 46.23 16.98 20.75 16.04 Students were asked to indicate the types of university-sponsored sales competitions in which they had competed. These results are shown in Table 17. Table 17 Competition Experience Students Competition Type N % Internal 20 25.00 External 30 37.50 None 30 37.50 Note. Students were able to select both internal and external. Perceptions regarding the three overall program learning outcomes were included in the survey. Table 18 shows the results for all respondents in all sales courses. The introductory and advanced level course results were compared and are shown in Table 19. For the introductory course results, only those students who had not yet taken any of the other sales courses were included. All questions utilized a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. 78 Table 18 Sales Program Outcomes Agreement with Statement Mean Std Dev I understand the sales process 4.17 0.76 I understand sales concepts and how to apply them 3.98 0.96 I know how to structure a sales presentation Note. n = 63. 3.95 0.95 As shown in Table 19, there was an observed difference in the mean scores between the introductory (M=3.85, SD=0.78) and the advanced course (M=4.43, SD=0.42) for understanding the sales process, with the advanced course students showing significantly higher agreement. The two-sample t-Test demonstrated a significant effect for agreement, t(39,17) = -2.50, p = .008. There was an observed difference in the mean scores between the introductory (M=4.00, SD=0.63) and the advanced course (M=4.60, SD=0.40) for understanding sales concepts and how to apply them, with the advanced course students showing significantly higher agreement. The two-sample t-Test demonstrated a significant effect for agreement, t(39,17) = -2.62, p = .006. The t-Test demonstrated no significant effect for knowing how to structure a sales presentation, t(39,17) = -1.35, p = .091. The difference between mean ratings among the introductory course students (M = 4.05, SD = 0.73) and advanced course students (M = 4.40, SD = 0.69) were not significant. 79 Table 19 Sales Program Outcomes: Introductory and Advanced Sales Course Comparison Agreement with Statement Mean Introductory Advanced Sales Sales I understand the sales process I understand sales concepts and how to apply them 3.85 4.43** 4.00 4.60** I know how to structure a sales presentation 4.05 4.40 t(39,17) p -2.50 .008 -2.62 .006 -1.35 .091 Note. n=39 Introductory Sales, n=17 Advanced Sales. *Significant difference between Intro and Advanced Sales courses at p=.05 ** Significant difference between Intro and Advanced Sales courses at p=.01 Students were asked to rate their agreement with statements about the selling skills taught in the sales program. All questions utilized a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. Table 20 shows these results by course. The two-course series of the introductory level and the advanced level sales course were directly compared using a twosample t-Test of means. The sample for the introductory sales course included only those students who had not yet taken the advanced sales course. The sample for the advanced sales course included students who had completed both the introductory and the advanced courses. As shown in Table 20, there was an observed difference in the mean scores between the introductory (M=4.05, SD=0.99) and the advanced course (M=4.63, SD=0.62) for sales presentations, with the advanced course students showing significantly higher agreement, t(39,16) = -2.13, p = .019. There was an observed difference in the mean scores between the introductory (M=3.95, SD=0.76) and the advanced course (M=4.44, SD=0.73) for rapport building, with the advanced course students showing significantly higher agreement, t(39,16) = 80 -2.19, p = .016. There was an observed difference in the mean scores between the introductory (M=4.11, SD=0.69) and the advanced course (M=4.56, SD=0.51) for consultive communication, with the advanced course students showing significantly higher agreement, t(39,16) = -2.39, p = .010. There was an observed difference in the mean scores between the introductory (M=3.97, SD=1.04) and the advanced course (M=4.56, SD=0.51) for dealing with objections, with the advanced course students showing significantly higher agreement, t(39,16) = -2.15, p = .018. There was an observed difference in the mean scores between the introductory (M=3.77, SD=0.74) and the advanced course (M=4.50, SD=0.52), for gaining commitment for the sale, with the advanced course students showing significantly higher agreement, t(39,16) = -3.59, p = <.001. There was an observed difference in the mean scores between the introductory (M=3.62, SD=0.75) and the advanced course (M=4.81, SD=0.54) for collaboration and teamwork, with the advanced course students showing significantly higher agreement, t(39,16) = -5.79, p = <.001. There was an observed difference in the mean scores between the introductory (M=3.21, SD=0.98) and the advanced course (M=4.50, SD=0.73) for negotiation, with the advanced course students showing significantly higher agreement, t(39,16) = -4.77, p = <.001. The t-Test demonstrated no significant effect for asking questions, t(39,16) = -1.58, p = .060. The difference between mean ratings among the introductory course students (M = 4.67, SD = 0.48) and the advanced course students (M = 4.88, SD = 0.34) were not significant. The tTest demonstrated no significant effect for effective listening, t(39,16) = -1.07, p = .144. The difference between mean ratings among the introductory course students (M = 4.67, SD = 0.48) and the advanced course students (M = 4.81, SD = 0.40) were not significant. Mean scores for those who had taken the negotiation course were all between 4.33 and 4.89. Mean scores for the sales technology course ranged from a low of 3.94 for negotiation 81 skills to a high of 4.35 for sales presentation skills. There was no pre-requisite sales course for these two courses so no comparison of mean scores was performed. Table 20 Mean Results for Selling Skills The sales course(s) helped me build selling skills in: Introductory Sales Advanced Sales Negotiation Sales Technology Sales Presentations 4.05 4.63* 4.33 4.35 Rapport Building 3.95 4.44* 4.39 4.06 Consultive 4.11 4.56* 4.72 4.24 communication Dealing with 3.97 4.56* 4.56 4.12 objections Gaining commitment 3.77 4.50** 4.56 4.12 Collaboration and 3.62 4.81** 4.72 4.24 Teamwork Negotiation 3.21 4.50** 4.72 3.94 Asking Questions 4.67 4.88 4.83 4.41 Effective listening 4.67 4.81 4.89 4.35 Note. n=39 Introductory Sales, n=16 Advanced Sales, n=22 Negotiation, n=19 Sales Technology. *Significant difference between Intro and Advanced courses at p=.05 ** Significant difference between Intro and Advanced courses at p=.01 Students were asked to rate their agreement with six statements about self-management and customer relationship management skills or knowledge learned in the sales courses. All questions utilized a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. Table 21 shows these results by course type. The two-course series of the introductory level and the advanced level sales courses were directly compared using a two-sample t-Test of means. The sample for introductory sales included only those students who had not yet taken advanced 82 sales. The sample for advanced sales included students who had completed both the introductory and the advanced courses. There was an observed difference in the mean scores between the introductory (M=3.07, SD=1.33) and the advanced course (M=3.67, SD=0.95) for sales technologies, with the advanced course students showing significantly higher agreement, t(39,16) = -1.75, p = .043. There was an observed difference in the mean scores between the introductory (M=4.33, SD=0.62) and the advanced course (M=4.73, SD=0.46) for problem solving, with the advanced course students showing significantly higher agreement, t(39,16) = -2.26, p = .014. There was an observed difference in the mean scores between the introductory (M=3.36, SD=0.99) and the advanced course (M=4.07, SD=1.16) for dealing with ethical issues, with the advanced course students showing significantly higher agreement, t(39,16) = -2.25, p = .014. The t-Test demonstrated no significant effect for professionalism, t(39,16) = -0.87, p = .193. The difference between mean ratings among the introductory course students (M = 4.74, SD = 0.50) and advanced course students (M = 4.87, SD = 0.35) were not significant. The t-Test demonstrated no significant effect for networking, t(39,16) = 0.83, p = .206. The difference between mean ratings among the introductory course students (M = 4.72, SD = 0.46) and the advanced course students (M = 4.60, SD = 0.51) were not significant. The t-Test demonstrated no significant effect for time management, t(39,16) = -1.54, p = .065. The difference between mean ratings among the introductory course students (M = 4.23, SD = 0.84) and the advanced course students (M = 4.60, SD = 0.63) were not significant. Mean scores for those who had taken the negotiation course ranged from a low of 3.56 for sales technologies to a high of 4.72 for problem solving. Mean scores for the sales technology course ranged from lows of 4.28 for networking with others to highs of 4.62 for problem solving 83 and professionalism. There was no pre-requisite sales course for these two courses so no comparison of mean scores was performed. Table 21 Mean Results for Self-Management and Customer Relationship Management The sales course(s) helped me build skills in: Introductory Sales Advanced Sales Negotiation Sales Technology Sales Technologies 3.07 3.67* 3.56 4.33 Problem Solving 4.33 4.73* 4.72 4.62 Dealing with Ethical 3.36 4.07* 4.39 4.33 Issues Professionalism 4.74 4.87 4.83 4.62 Networking 4.72 4.60 4.39 4.51 Time Management 4.23 4.60 4.33 4.28 Note. n=39 Introductory Sales, n=16 Advanced Sales, n=22 Negotiation, n=19 Sales Technology. *Significant difference between Intro and Advanced Sales courses at p=.05 Seven questions to gauge perceptions of the selling profession were included in the survey. The first three questions were positively worded and the other four questions were negatively worded. These questions were included to assess perceptions after completing the introductory course, and if those perceptions changed as students were exposed to courses beyond the introductory level. Table 22 shows the mean results by course type. All questions utilized a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. The twocourse series of the introductory level and the advanced level sales course were compared using a two-sample t-Test of means. The sample for the introductory sales course included only those students who had not yet taken the advanced sales course. The sample for the advanced sales 84 course included students who had completed both the introductory and the advanced courses. No significant differences were observed between the introductory and the advanced course results. Table 22 Mean Results for Perceptions of the Selling Profession Salespeople Are respected by others Introductory Sales Advanced Sales Negotiation Sales Technology 3.74 3.53 3.61 3.40 Are admired by others 4.08 4.13 4.33 Are perceived favorably by others 3.77 3.93 3.83 Take advantage of uneducated 3.21 2.87 2.94 buyers Stretch the truth to make a sale 3.23 2.87 2.78 Make something up when they do 2.85 2.47 2.39 not know the answer to a question Misrepresent guarantees and/or 2.87 2.73 2.89 warranties Note. n=39 Introductory Sales, n=17 Advanced Sales, n=22 Negotiation, n=19 Sales Technology. 3.87 3.40 2.93 3.27 3.00 3.27 Three questions to gauge perceptions about a selling career were included in the survey. These questions were included to assess perceptions after completing the introductory course, and how those perceptions may change as students are exposed to courses beyond the introductory level. Table 23 shows the mean results by course type. All questions utilized a 5point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. The two-course series of the introductory level and the advanced level sales course were compared using a two-sample t-Test of means. The sample for the introductory sales course included only those students who had not yet taken the advanced sales course. The sample for the advanced sales course included students who had completed both the introductory and the advanced courses. No significant differences were observed between the introductory and the advanced course results. 85 Table 23 Mean Results for Perceptions about a Selling Career Course Advanced Negotiation Sales 4.53 4.56 Introductory Sales A sales career/selling Sales Technology Gives a sense of accomplishment 4.36 4.07 Is doing something worthwhile on 4.18 4.40 4.33 4.07 the job Is personally satisfying 4.21 4.40 4.28 4.00 Note. n=39 Introductory Sales, n=17 Advanced Sales, n=22 Negotiation, n=19 Sales Technology. Students were asked to state their interest in pursuing a sales career after graduation. Results are shown in Table 24. This question utilized a 5-point scale where 1 = not at all interested and 5 = very interested. Nearly 62% of students overall and 60% of those who had taken only the introductory course rated their interest as a 4 or 5. Just under 71% of those who had completed the advanced sales course rated their interest as a 4 or 5. Table 24 Interest in a Sales Career All Respondents n % 3 4.76 8 12.70 13 20.63 19 30.16 20 31.75 Introductory Sales n % 2 5.13 6 15.38 8 20.51 13 33.33 10 25.64 Advanced Sales Interest in Sales Career n % 1=Not at all interested 0 0 2 0 0 3 3 17.65 4 4 23.53 5 = Very interested 8 47.06 No Response 2 11.76 TOTAL 63 100 39 100 17 100 Note. Negotiation and Sales Technology counts are included in All Respondents To gauge student interest in current or potential future sales program features, students were asked to rate the importance of six opportunities. This question utilized a 5-point Likert- 86 type scale ranging from 1 = very important to 5 = very unimportant. Results in Table 25 indicated the strongest interest in involvement from sales professionals, followed by the opportunity to earn a certification in sales, facilities for role plays and video recording, participation in competitions, a sales minor, and a sales major. Table 25 Mean Results for Importance of Sales Program Features Benefit or Opportunity Mean Std Dev Involvement from Sales Professionals 4.47 0.60 Nationally recognized sales certification available 4.36 0.76 Role Play and other Video recording Facilities 4.32 0.80 Participating in Competitions Ability to earn a Sales MINOR Ability to earn a Sales MAJOR Note. n = 63. 4.28 4.25 4.07 0.90 0.79 0.94 Sales Program Evaluation, Part 4 of the Methodology Part 4 was based on the Kirkpatrick model for the evaluation of training and learning programs. Sales program results were assessed using four levels of criteria: reaction, learning, behavior, and results. Level 1, Reaction, showed the results for the student evaluations of instruction for sales courses and was used as part of the overall program evaluation. Level 2, Learning, summarized mastery of skills utilized in role play scenarios. Level 3, Behavior, examined the results from end of course projects in two courses, and student internship results. The Level 2 and 3 results were used to evaluate sales competencies. Level 4, Results, showed the outcomes measures of job placement percentage, average starting salary, and job titles for students who had recently graduated from the sales program. 87 Level 1 Reaction The results of the student evaluation of instruction for the sales courses are shown in Table 26. These results represented the aggregated sales program course evaluation results from the spring 202, and fall 2021 semesters. Using the standard university course evaluation instrument and process, students evaluated each course on the attributes listed using a 5-point Likert scale where 5=strongly agree, 4=agree, 3=neither agree nor disagree, 2=disagree, and 1=strongly disagree. Mean scores were used to assess student affective reaction and utility judgments on the criteria rated for each course. All four of the courses showed very positive results, with mean ratings at 4.5 or higher among all attributes for the three advanced courses, and all attributes above 4.2 for the introductory sales course. Exceptionally high ratings on several questions are noteworthy. For the introductory course, high scores were recorded for instructor mastery of the subject matter and instructors behavior reflecting concern for learning success. For the advanced course, instructors behavior reflecting concern for learning success and recommendation of instructor to another student were exceptionally high. In the negotiation results, areas with the highest scores were instructor mastery of subject matter, instructors behavior reflecting concern for learning success, and recommendation of instructor to another student. Prompt and useful feedback, instructor availability, and recommendation of instructor to another student were the areas with the highest scores. 88 Table 26 Student Evaluation of Instruction Results Attribute Introductory Sales Std M Dev Advanced Sales Std M Dev Negotiation Std M Dev Sales Technology Std M Dev Syllabus communicated 4.42 0.63 4.5 0.58 4.86 0.38 4.82 0.26 learning outcomes Readings, discussions, lectures, projects helped me 4.58 0.52 4.5 1.00 4.86 0.38 4.69 0.37 attain learning outcomes Multiple instructional 4.52 0.52 4.5 0.58 4.86 0.38 4.69 0.26 methods were used Activities and assignments 4.42 1.03 4.75 0.50 4.86 0.38 4.75 0.27 supported learning outcomes Activities and assignments challenged me to think more 4.58 0.97 4.5 1.00 4.86 0.38 4.63 0.36 deeply/critically about the course subject matter Would recommend course to 4.23 1.42 4.75 0.50 4.86 0.38 4.50 0.52 another student Instructor demonstrated 4.69 0.53 4.75 0.50 5.00 0.00 4.75 0.35 mastery of subject matter Instructor provided well4.54 0.79 4.75 0.50 4.86 0.38 4.69 0.37 organized learning activities Instructor provided prompt, 4.34 1.14 4.75 0.50 4.86 0.38 4.82 0.37 useful feedback Instructor was available on a 4.23 0.95 4.75 0.50 4.86 0.38 4.82 0.37 regular basis Instructor behavior clearly reflected genuine concern for 4.64 0.7 5.00 0.00 5.00 0.00 4.75 0.53 my learning success I would recommend 4.39 1.10 5.00 0.00 5.00 0.00 4.82 0.37 Instructor to another student Note. n=18 Introductory Sales, n=4 Advanced Sales, n=7 Negotiation, n=14 Sales Technology n=14. 89 Level 2 Learning This evaluation utilized a pretest-posttest design for the introductory level and the advanced level sales courses. Five sales concepts measuring the nine core selling skills were evaluated in each course using role play scenarios. The maximum points for each concept were 10 and the maximum points for the overall score was 50, or the sum of the points earned. The analyses included the following: 1. Mean and standard deviation for each concept; 2. Percent of students achieving the mastery level; 3. Within group comparison, pre- and post-intervention, using a paired sample t-Test where pre and post level was nominal and total points earned on the rubric was interval level; 4. Between group comparison of end of semester scores for the introductory versus advanced course, using an independent samples t-Test. The evaluation scale used was Mastery (10 9), Partial Mastery (8 7), Progressing (6 5), and Emerging (<4). The introductory course results are shown in Table 27. Two sections of the course, one in the spring of 2021 and one in the fall of 2021 were included in the total for this analysis. The courses were taught by two different instructors. The rubrics between semesters had slight wording differences. The five concepts evaluated showed a significant increase in mastery between the initial role play and the final role play evaluation. By the end of the semester, mastery levels ranged from a low of 43% of students for building trust and listening skills to a high of 80% of students for the sales approach, which measured the sales process, professionalism, and rapport building skills. 90 Table 27 Introductory Sales Course; Within Group Role Play Evaluation Introductory Course Initial Role Play Final Role Play Concept M M SD t(42) p SD Sales Approach 6.82 1.14 9.43 0.12 -13.65 Building Trust 6.74 1.00 8.62 0.14 -12.40 Effective Questioning 6.75 1.04 8.78 0.12 -13.40 Product Presentation 6.68 0.94 8.61 0.14 -15.05 Obtain Commitment 6.64 1.03 8.91 0.13 -16.28 Note. Paired t-Test of means (one-tailed). *** Significant difference between initial and final role plays at p<.001 <.001*** <.001*** <.001*** <.001*** <.001*** % students achieving mastery 81 43 55 50 52 The advanced course results are shown in Table 28. One section of the course from the fall of 2021 was included in this analysis. The rubric used the language of overcoming objections, unlike the introductory course rubric using the terminology of building trust. All five skills evaluated showed a significant increase in skill level between the initial role play and the final role play evaluation. By the end of the semester, mastery levels ranged from a low of 50% for obtaining commitment, which measured skills in persuasion and solving customer problems, to a high of 80% for the sales approach, which measured the sales process, professionalism, and rapport building skills. 91 Table 28 Advanced Sales Course; Within Group Role Play Evaluation Advanced Course Concept Initial Role Play Final Role Play M M SD t(10) p SD Sales Approach 6.60 1.65 8.90 1.10 -5.44 Trust/ Overcoming Objections 6.80 1.55 8.80 0.79 -6.00 Effective Questioning 6.70 1.06 8.70 0.95 -9.49 Product Presentation 6.90 1.45 8.90 0.99 -7.75 Obtain Commitment 6.60 0.65 8.40 1.07 -4.32 Note. Paired t-Test of means (one-tailed). *** Significant difference between initial and final role plays at p<.001 <.001*** <.001*** <.001*** <.001*** <.001*** % students achieving mastery 80 60 60 70 50 A between group comparison of the end of semester scores for the introductory and advanced courses was performed using an independent samples t-Test (two-tailed). These results are shown in Table 29. No significant differences were observed between the introductory and the advanced end-of-course role play comparison results. Table 29 Introductory and Advanced Course Between Group Final Role Play Comparison Introductory Concept Approach Building Trust/Overcoming Objections Effective Questioning Product Presentation Obtain Commitment Advanced M SD M SD t (42,10) 9.43 8.62 8.78 8.61 8.91 0.12 0.14 0.12 0.14 0.13 8.90 8.80 8.70 8.90 8.40 1.10 0.79 0.95 0.99 1.07 1.76 -0.58 0.27 -0.92 1.64 p 0.08 0.57 0.78 0.36 0.11 Level 3 Behavior This evaluation consisted of the end of course project in two courses and supervisor evaluations of student internships. 92 End of Course Projects. The end of course final project data were gathered using rubrics designed for evaluating the final projects in the two 400-level courses offered. In the advanced course, students were evaluated by industry experts and sales program faculty members. For the negotiation course, students were evaluated by a part-time faculty member and an external sales executive. For the advanced sales course, the end of course project used for evaluation was a national sales competition where the ten students in the course competed against six other nationally recognized sales programs. Students worked on one or more parts of the competition, including the pitch preparation, research, marketing, financial analysis, and live pitch to the judges. Each university team was evaluated and ranked by the industry sales experts serving as competition judges. The student team being studied for this project was ranked as number two by the external judges, against a total of seven teams. The competition scores were not able to be obtained. Four of the six other teams were recognized as USCA full member programs. Four of the six teams had 120 or more sales students each. One school reported 75 students and one reported 30 students. Using the competition rubric, the student team was evaluated and scored by the course instructor and a second part-time faculty member in the sales program. These scores were averaged and are reported in Table 30. The rubric used a scale of 1 to 5, with a 4.5 or above considered at the Mastery level and 3.5 to <4.5 considered to be Partial Mastery. The students were rated as achieving mastery in their approach, presentation of the solution, overcoming objections, and their action plan. Students were rated as achieving partial mastery in confirmation of needs and sales technology and analytics. 93 Table 30 Advanced Sales End of Course Team Project Evaluation Advanced Course Mean Score Mastery Observed Approach/Took the Lead 5.00 Yes Confirmation of Needs 4.25 Partial Technology & Analytics 4.25 Partial Communication/Presentation of Solution 5.00 Yes Overcame Objections 4.75 Yes Developed Action Plan 4.5 Yes Note. Mean score computed as the average of two instructor evaluations for the team. For the negotiation course, the end of course project used for evaluation was a final culminating negotiation project where each of the 20 students in the course participated as an individual negotiator against another student in the course. Students were assigned randomly. Scores were obtained for each student using a rubric to evaluate five different concepts on a 10 point scale, where 9 10 was considered mastery of the concept, 7 8 was considered partial mastery, 6 was considered emerging, 5 was considered still developing, and less than five was considered weak or limited use of the concept. The mean results are shown in Table 31. Fortyfive percent of the students were judged to have achieved mastery (n=9) and 40% achieved partial mastery (n=8). Two students were judged to have demonstrated skills that are emerging and one student was not able to demonstrate the needed skills and was judged as still in development. 94 Table 31 Negotiation End of Course Project Evaluation Negotiation Course Concept Evaluated Preparation Understanding Issues Questioning Generated Options/Solving Problems Built Trust Note. n=20 Mean Score out of 10 7.65 7.95 7.50 7.95 7.50 Sales Internship. Each student completing a sales internship was evaluated by their supervisor on communication and teamwork. Communication was one of the nine core selling skills in the selling skills framework established for the sales program being evaluated. Teamwork was one of the ten additional skills in the framework. The questions utilized a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. Fifteen sales program students were evaluated and their scores were compared to all 288 business students completing any business internship. Sales students were evaluated as performing well for both communication and teamwork skills, with both mean ratings at 4.60 or higher. While directionally higher than the results for all business students, there was no significant difference between the two groups. The results are shown in Table 32. Table 32 Sales Internship Supervisor Evaluation Attribute Mean Sales All Business Students Students Communication 4.60 4.52 Teamwork 4.69 4.61 Note. n=15 sales students; n=288 for all business students 95 Level 4: Results The fourth level of the evaluation consisted of obtaining and reporting on three student outcomes metrics. The three outcomes measures were the following: 1. Job placement rate, defined as obtaining full-time, professional placement in a business role within six months of graduation, as reported by the universitys career center. 2. Starting Salary, self-reported by the graduate in the student exit survey. 3. Job Title, as self-reported by the graduate in the student exit survey. Comparisons were made to all graduates of the business school as well as industry averages from the National Association of Colleges and Employers (NACE) First Destination Dashboard (2020). Six students completed the program and graduated with the sales specialization in 2021. No students dropped out of the program. All six students were male. The job placement rate was 100%. The median starting salary for these students was $65,750 and the mean was $75,028. Comparison data for job placement and starting salary are shown in Table 33. All six graduates obtained a sales-related professional position. The job titles they reported were: 1. Account Manager 2. Account Specialist 3. Assistant District Manager 4. Associate, Commercial Real Estate 5. Associate District Manager 6. Entrepreneur, Real Estate 96 Table 33 Career Outcomes and Comparison Outcome Job Placement % Mean Starting Salary $ a Sales Program Graduates (n=6) Universitys Business School Graduates (n=70) National Business School Graduates 2020 (n=95,233) 100 100 87a 75,028 53,299 55,572 NACE Class of 2020 Graduate Outcomes Report. The placement percentage includes employed, continuing education, military, and anyone else not seeking employment. 97 Chapter 5: Discussion and Evaluation of the Capstone Project Summary Summary of the Project and Significance of the Outcome This capstone project addressed the research question: What knowledge and skills do students need in preparation for a professional selling career, and how does implementing these in a formal curriculum improve students' sales competencies? There was some prior research on the knowledge and skills needed by business graduates in preparation for a sales career but nothing specific to small school programs. Research on measurement or evaluation of skill attainment had centered on specific courses or pedagogies rather than an entire curriculum. Addressing a comprehensive sales program development and evaluation process relative to a small university was nonexistent in the literature. The start of this project centered on determining the skills needed by sales graduates to develop sales competency and the curriculum required to deliver this result. Once the new curriculum was defined and implemented as a new program in professional selling, the second part of the project, an evaluation of the new program, was performed. The newly developed curriculum consisted of a nine credit hour specialization in professional selling. This new specialization was designed so that all students completed an introductory selling course, an advanced selling course, and an additional elective sales course from a list of options. To address the question of what selling skills were needed, a rigorous process was completed. It included an extensive literature review, including documentation from SEF and the USCA, interviews with experts who ran successful university sales programs, sales executives who routinely hired new graduates for sales roles, and an executive from an organization dedicated to sales research in universities. This process identified nine core skills 98 required to build sales competency. These core skills were introduced and practiced in the introductory level sales course. The core skills were reinforced in the advanced level course. Ten additional skills were identified as useful skills and these were introduced in the advanced level course. Discussions with internal marketing and sales faculty were used to confirm the courses to be included in the program, the skills and pedagogies to be incorporated in the sales coursework, along with how the skills would be introduced, reinforced, and assessed throughout the program. Following the initial program implementation, a quantitative survey was completed with all students in the program to assess their perceptions. Survey results suggested a high level of satisfaction among the students in the program and the perception that they had increased their competency in sales knowledge and skill attributes. This student survey was implemented prior to the formal evaluation of learning and was used as an early indicator of the program design. Student perceptions of satisfaction were later verified as part of the evaluation process using the universitys course analysis results. These results were part of the Level 1, Reaction, measures. The second part of the project was the formal evaluation of sales competency, defined as the ability of the students to implement the nine core skills. This was measured by a modified Kirkpatrick framework using Level 2 and 3 criteria, as explained in Table 10. This evaluation occurred over a three-semester period. A significant increase in sales competency was observed using this evaluation framework. This increase was measured by 1) within group knowledge and skill development gains using a pretest; post-test role play exercise in the introductory and advanced sales courses; 2) end of course project evaluations in two courses; and 3) internship supervisor evaluations. Further supporting this evidence were the career outcomes among program graduates. The conclusion was that the new professional selling program did provide the knowledge and skills students needed to build sales competency in preparation for a selling 99 career. Given the lack of research in this area, along with the strong market demand for graduates with selling skills, the result of this project was noteworthy. It contributed to the growing body of sales research, it documented a program design and evaluation process, and it provided supporting documentation for both students pursuing a selling career and businesses that require sales graduates. Discussion of the Knowledge and Skill Requirements Quantitative Survey For the nine core skills taught in the program, six showed a significant increase in perceived skills level between the introductory and advanced sales students on the initial quantitative survey of students in the sales program. The three core skills that did not show a significant difference, professionalism, asking questions, and effective listening, were directionally higher among the advanced students. A possible explanation for these results was the program design itself, which incorporated a substantial time commitment and focus on ensuring the attainment of these three skills at the introductory course level. Supporting this explanation were the mean and top box scores. These three skills had the highest mean values among the introductory students at 4.67 on a 5-point scale for asking questions and effective listening and 4.74 on a 5-point scale for professionalism. In addition, 100% of the introductory student responses indicated they agreed or strongly agreed that they had developed all three of these skills by the end of the introductory course. The advanced sales course students reported higher perceived skills attainment versus introductory students in 14 of the 15 skill areas on the survey, with 10 of those differences being statistically significant. While the student perception survey was not part of the formal course evaluation procedure and was not indicative of true learning, the results were valuable in understanding student perceptions of sales competency growth between the introductory and advanced level 100 course. The survey results also provided an initial benchmark for perceived attainment of the selling skills taught in the program and the three program learning objectives. As shown in Chapter 4, Table 19, student perceptions around the three program learning objectives were positive, with scores of 3.85 or higher on a 5-point Likert scale for agreement with meeting each objective. To further support the positive perceptions of meeting the program objectives, top box scores were assessed. Top box scores are defined in this paper as the sum of percentages for the top two points, such as strongly agree and agree, on a five-point scale. For the introductory course, 85% of respondents agreed or strongly agreed that they understood sales concepts, 85% agreed or strongly agreed they understood the sales process, and 76% agreed or strongly agreed that they knew how to structure a sales presentation. Only 3% disagreed with understanding sales concepts and the sales process, while 7% disagreed with knowing how to structure a sales presentation. Top box scores for the students who had completed both the introductory and the advanced course showed even more positive results, with 93% agree or strongly agree responses to sales concepts and the sales process. These results showed a statistically significant difference between the introductory and advanced level students (p=.01), with students at the advanced level reporting higher agreement. Top box scores among advanced level students for the third program objective were 94% agree or strongly agree for knowing how to structure a sales presentation. For all three of the program objectives, all remaining responses from the advanced students were in the neutral category and no students disagreed with any of the three program objectives statements. While these results suggest that the program has been successful in meeting its learning objectives, assessing these perceptions among the introductory level course cohort once they have completed the advanced course could provide stronger insight as to perceived growth among students who are progressing through the program. 101 It should be noted that the survey results were based on student perceptions and not actual skill attainment. For this reason, it was difficult to draw any specific conclusions based on these results alone. The findings were valuable as an initial check on course content rather than for the formal evaluation. The results suggested that all courses were perceived favorably. If repeated in the future, the survey results may serve as a valuable benchmark against which to measure changes in student perceptions. Additional questions were included in the student survey to gain insight into the future needs of the program. These results were reported in Chapter 4. They were not perceived to be of specific value to the knowledge and skill identification process and were therefore not addressed further. Discussion of the Program Evaluation Results The Level 2 and 3 evaluation results were used for the formal program evaluation. They provided the framework to measure sales competency among students in the new professional selling program. The results of this evaluation showed that the implementation of the nine core skills in the formal curriculum did improve students sales competencies. Level 2, Learning, was used to evaluate the introductory and advanced sales courses using a pre-test, post-test within group role play in each course. The use of role play is regarded as one of the most popular and effective methods to teach sales related academic curricula by both professors and sales representatives (Parker, Pettijohn, & Luke, 1996; Sellars, D. 2005). The scoring rubrics covered the sales process by evaluating students on five concepts that encompassed the nine core skills. Both the introductory and advanced course students showed significant growth (p<.001) on all five sales concepts using paired sample t-Tests, demonstrating improvement in sales competency between the pre to the post-test. Every student in both courses 102 showed growth between the pre and post-test, with the exception of one student in the advanced course who scored at the highest level, or mastery, on both the pre and post-test. All students performed at the mastery or near mastery level in each category by the end of the course. It is unlikely that the growth observed between the pre and post-test could be due to the mere practicing of a role play experience. The role plays used for the pre and post-tests were unique scenarios that required the application of the complex core skills that had been taught and reinforced throughout the semester. Application of these skills to each unique scenario was required to demonstrate mastery in the role play experience. The use of multiple evaluators, including qualified faculty and industry professionals, provided further evidence for the objectivity of the results. The Level 2 evaluation also included a between group comparison of the final role play results in the introductory and advanced courses. This was performed using an independent samples t-Test. There was no significant difference observed on any of the five concepts when comparing the final role play scores in the introductory course to the final role play scores in the advanced course, as shown in Table 29. There are several possible explanations for this result. It is possible that there was no observed increase in sales competency between the introductory course and the advanced course. Supporting this explanation was the similarity in initial role play scores between the two groups. A comparison of the between group initial role play scores in each course indicated a similar starting point among both groups, as shown in Tables 27 and 28. In fact, a mean of means for the five concepts that were evaluated showed an introductory course initial role play value of 6.73 and a nearly identical advanced course value of 6.72. Interpreted on its own, this could point to learning loss between the conclusion of the introductory course and the beginning of the advanced course. However, this conclusion assumes 103 that the two initial role play scenarios used, different for each course, were of a similar challenge level. This was not the case. The introductory course role plays used for both pre and post-test were beginner level role plays with a focus on information gathering skills such as questioning and listening. The advanced course role plays, both initial and final, were significantly more challenging, commensurate with an advanced level sales course. While the role play rubrics measured the same core skills, the application of these skills was more difficult in the advanced course role plays due to more complex scenarios. In addition, the advanced course role play scenarios were designed with less transparency surrounding the desired role play outcome than what was provided in the introductory course scenarios. For these reasons, learning loss between courses was not believed to be the most likely explanation for the lack of between group sales competency growth. A better explanation was that a between group comparison was not an appropriate measure due to the difference in the level of difficulty between the role play scenarios. Therefore, the conclusion is that a between group comparison measure is not valid for the data collected in this study. True learning among one student cohort, or a within group measurement from one course to the next may be a more reliable indicator of growth. This measure was not able to be assessed within the time frame of this study. A within group evaluation of growth from the conclusion of the introductory course to the conclusion of the advanced course, once the current students have progressed to the advanced course, is recommended as a future measure of competency. Level 3, Behavior, included two different measures. End of course projects were evaluated in the advanced sales course and the negotiation course. The advanced sales course students were evaluated by industry sales experts as part of a national sales competition. They ranked second among seven nationally-recognized sales programs, placing just behind a program 104 that was typically ranked in the top three nationally on any metric and has graduated more than 100 students majoring in professional selling annually for many years. When judged internally using the same rubric, the students in the course achieved mastery, or a score of five, on four concepts covering 14 skills. They achieved near mastery, or a score of 4.25 on the five-point scale, for the two concepts dealing with skills in interpretation of needs and use of technology and analytics. These two skills were introduced for the first time in the advanced course. The competition ranking among industry experts and in comparison to other established programs, along with the internal evaluation using the competition rubric, suggests sales competency among the advanced course students. Competition organizers typically provide video of the teams participation following the competition. Use of this video for further evaluation by businesses that recruit from the school in this study is recommended as a future metric for judging sales competency among the advanced course students. Although not all students in the program completed the elective course in negotiation, eight of the nine core skills were reinforced in this course, along with advanced skills related to negotiation. For this reason, the end of course project in the negotiation course was evaluated as part of the Level 3, Behavior, measure. The end of course project was a negotiation role play scenario. Five concepts covering the core and advanced skills were evaluated by the course instructor and an industry sales executive using a rubric. Forty-five percent of the students achieved the mastery level overall and the remainder achieved partial mastery. Negotiation requires a combination of advanced level skills, several of which are not introduced in the introductory course. Therefore, many of these students were exposed to these negotiation concepts for the first time in this course. The outcome obtained was believed to be acceptable for a newly developed course and aligned with the goals and objectives of the selling program; 105 however, no mastery goal was initially set for this course. This initial result should be used to evaluate the proportion of students achieving mastery in future semesters. As the only course where negotiation skills are the primary focus, continuing to track and evaluate this metric is important and will help in understanding student mastery of negotiation skills. Student internships were evaluated as another part of the Level 3 measurement. All students formally enrolled in the sales specialization and who had completed any internship following their junior year with sales as a responsibility were included in the analysis. Each internship supervisor provided the evaluation at the end of the students internship. To minimize differences in inter-rater agreement among the supervisors, each was provided with a standardized rubric for the evaluation and instructions for its completion. Although inter-rater agreement is not validated further, these data were valuable in that they provided an external evaluation measurement from individuals who were not closely affiliated with the sales program or its faculty and were therefore unlikely to be influenced by such an association. Communication, one of the five attributes measured by the internship supervisor, was one of the nine core selling skills. The mean result for this was 4.60 on the five-point scale, indicating that communication was a strength among the sales students who had completed an internship. In comparison, communication was rated at 4.52 for all business students. While statistically this difference was not significant, it was insightful that the sales student score was slightly higher. Teamwork, another one of the five attributes measured by the internship supervisor, was one of the ten additional selling skills introduced in the advanced sales course. The mean result for this attribute among the sales students was 4.69 on the five-point scale. This indicated that teamwork skills were a strength for sales students who had completed an internship. In 106 comparison, the mean result for teamwork among all business student interns was 4.61. While directionally higher, the difference was not statistically significant. The small sample size of sales students (n=15) may have been a contributor to the lack of significance in the result. As more students enter the sales program and complete sales internships, these data should be reexamined for significance and additional insight into sales competency judgments. The level 2 and 3 results provide evidence to suggest that there was an observed increase in sales competency among the sales program students. Therefore, the conclusion was made that the program did provide the knowledge and skills students need in preparation for a successful selling career, and implementing these skills in a formal curriculum did improve sales competency. The Level 1 and Level 4 measures did not directly assess sales competency but were identified as important in the program evaluation process for the university. Level 1, Reaction, provided student perspectives of each course using the universitys course evaluation instrument. Student affective reactions indicated a high level of satisfaction with all four of the sales courses. Scores for all attributes measured, for every course, were well above 4.0 on a five-point Likert scale. The mean of means for the four sales courses combined was 4.63 (n=37), as compared to the mean of means for all business school courses of 4.45 and all courses at the university of 4.41. The number of respondents at the school and university level was unknown. Student satisfaction may be an important indicator of student persistence in the program and therefore would be worthwhile to track in the future. Level 4, Results, consisted of measures related to student outcomes six months postgraduation. While not directly measuring sales competency, these data were insightful in terms of student success in the short term and these results bode well for the sales program. This 107 information could potentially be monitored as a leading indicator of post-graduation success as these graduates progress in their careers. While only six students were included in the Level 4 results, all of them reported successful outcomes as of the six-month post-graduation measurement period. All achieved their goal of a sales-related professional position, reported job titles commensurate with a professional selling role, and starting salaries equal to or above the mean salary for all of the schools business school graduates (n=70) from May 2021. This result suggested that graduates of the sales program were obtaining high quality professional selling positions. The program evaluation results in total supported a positive outcome as a result of the design, development, and implementation of the professional selling program. Limitations As noted previously, there were limitations to this study. Studying a small population in a single university may limit the generalizability of the findings to other schools. The small sample size of students in this study, especially the small sample of students available for inclusion in the graduate outcomes salary results, should be considered when evaluating the findings relative to other universities or programs. The study results also relied on a limited number of instructors for the courses that were part of the evaluation. Evaluation of the Intervention This project generated a strong interest within the business school. The administration, faculty, and students were supportive of the initiative from the outset. This support was driven by the desire to provide a relevant curriculum that aligned with the needs in the marketplace and the needs of the student population in the business school. The administration recognized the value related to recruitment of new students, retention of current students, and engagement and 108 potential funding support from businesses. Students were enthusiastic about the job prospects in professional selling and the course content to prepare them for those future roles. This project also fulfilled a need among local businesses searching for new sales talent and the program has resulted in financial support from the business community. Leadership issues surrounding interactions with and support from local businesses created the most significant organizational issue during the project. Some early interest and financial support from businesses created leadership conflicts between departments. To facilitate the work of the project and ensure cross-functional and peer support, an understanding of the conditions and culture within other departments and between the business school and other groups was required. These challenges were addressed through the application of adaptive leadership principles. The researcher utilized these tools to develop the leadership capacity necessary to manage the process. Frequent meetings with collaborators and other stakeholders were required to understand sources of conflict and differing values and norms, clearly communicate needs and constraints, and evaluate limitations within the organization, its structures, and its processes. Implications and Reflection This project can be used by the university and the business school in several ways. It provides a model for designing, implementing, and evaluating new specializations in the curriculum. This includes not only the technical requirements for new program development but also in understanding the leadership implications that arise when there are multiple stakeholders and collaborators and not all share the same vision or values. The adaptive leadership model provides an opportunity to build leadership capacity among a broader group of individuals. 109 This project resulted in a positive experience for the stakeholders and collaborators involved. They were able to realize a successful outcome, as measured not only by the results of the project evaluation but in terms of additional goals held by these individual members. Namely, a primary goal that was outside of the project but very important to multiple stakeholders was to generate external funding for the program. In 2021, as external business partners became exposed to students in the program and saw success by students in national competitions, significant external funding was realized from multiple donors. Reflecting on the leadership challenges of the project provided additional insight. Specifically, the diagnostic process created the most significant challenge. It required collaboration and compromise. An important adaptive leadership tool was to build the emotional capacity to cede control in parts of the process. This required self-reflection about the role of the researcher as well as the role of the other stakeholders and their positions. The most critical tools used in the diagnosis were elegant questioning and maintaining a high level of curiosity. These skills allowed for better anticipation of difficulties and the ability to accept compromise when needed. Practice over many meetings built these capacities. Future Research Given the result of the between group role play comparison, further evaluation of this area using within group growth from the introductory to the advanced course would be of interest. An evaluation of the advanced course students using a role play scenario similar in difficulty level to the introductory course rubric would minimize the bias from the more challenging role play in the advanced course, as occurred in this project. Another area for follow up research would be to evaluate program graduates one year post graduation. Understanding their strengths and opportunities for development during their first year in sales would provide valuable insight into the program and its curriculum. 110 Feedback from direct supervisors of the new graduates would also be of value and would provide additional detail that could be relevant to specific sales roles or industries. Assessment of non-business students who complete a university sales curriculum, such as a minor or specialization is another area for future research. Program funding and financial support from industry were raised as a theme during the qualitative interviews. Gaining more detailed information on the needs and requirements of businesses that would be willing to support sales programs, as well as information about other potential funding sources, could be another subject for future study. 111 References AACSB (2020). 2020 Interpretive guidance for business accreditation standards. AACSB. https://www.aacsb.edu/accreditation/standards/business Adhabi, E. & Anozie, C.B. (2017). Literature review for the type of interview in qualitative research. International Journal of Education. 9(3) 1-12. Allen, C., Kumar, P., Tarasi, C. & Wilson, H. (2014). Selling sales: Factors influencing undergraduate business students decision to pursue sales education. Journal of Marketing Education. 36(2) 94-104. American Marketing Association. (2004). Revolution in selling and sales management education: Market-driven change in the classroom. AMA Summer Educators Conference. Anderson, N. & Douglas-Gabriel, D. (2022). Colleges scramble for students as enrollment plunges. The Washington Post. Anderson, R.E., Dixon, A.L., Jones, E., Johnston, M.W., LaForge, R.W., Marshall, G.W. & Tanner Jr., J.F. (2005). The Scholarship of Teaching in Sales Education, Marketing Education Review, 15:2, 1-10, DOI: 10.1080/10528008.2005.11488899 Arthur, W., Jr., Tubre, T. C., Paul, D. S., & Edens, P. S. (2003). Teaching effectiveness: the relationship between reaction and learning criteria. Educational Psychology, 23, 275 285. Barat, S., & Spillan, J.E. (2009). A cross country comparative analysis of students perceptions of the sales profession: A look at U.S., Peru, and Guatemala. Marketing Management Journal, 19 (2), 52-63. Belkin, D. (2020). Is this the end of college as we know it? The Wall Street Journal. 112 Bloom, B.S., Engelhard, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R., (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay. Bolander, W., Bonney, L., & Satornino, C. (2014). Sales education efficacy: Examining the relationship between sales education and sales success. Journal of Marketing Education, 39, 169-181. Bowe, J.J. (2015). Ethical decision making by future salespeople: When resolving sales dilemmas, do students in sales schools apply ethical philosophies differently from students in other business majors? Anderson University Doctoral Program. Bristow, D., Amyx, D., Castleberry, S. B., & Cochran, J. J. (2011). A Cross-Generational Comparison of Motivational Factors in a Sales Career Among Gen X and Gen-Y College Students. Journal of Personal Selling & Sales Management, 31(1), 77-85 Bristow, D.N., Gulati, R., Amyx, D. & Slack, J. (2006). An empirical look at professional selling from a student perspective. Journal of Education for Business, 81:5, 242-249, DOI: 10.3200/JOEB.81.5.242-249. Bureau of Labor Statistics Occupational Outlook Handbook (2019). Bureau of Labor Statistics. (2020, April 10). Occupational Outlook Handbook. https://www.bls.gov/ooh/management/home.htm. Accessed 10-26-20 Burke Institute. (2020). Seminars. https://www.burkeinstitute.com/ Bush, A. J., Bush, V. D., Oakley, J., & Cicala, J. (2014). Formulating undergraduate student expectations for better career development in sales: A socialization perspective. Journal of Marketing Education, 36, 120-131. 113 Castelberry, S. (2014). Salesperson ethics: An interactive computer simulation. Journal of Marketing Education 36(2), 209-216. Cespedes, F. V., & Weinfurter, D. (2016, April 26). More universities need to teach sales. Harvard Business Review. https://hbr.org/2016/04/more-universities-need-to-teach-sales Coco, C. M., Crumbly, J., & Jackson, F. H. (2014). The role of industry and university partnerships in developing significant learning outcomes related to business concepts. Journal of Academic Administration in Higher Education, 10(1), 1-5. Cox, B.T., Fyles, J., Nicholson, C. & Riggs, J. (2018). Professional Sales at Stetson University. Proceedings from the 7th Sales Educators Academy. May 16, 2018. Craig, R. (2020) How to close the skills gap when employers are from Mars, colleges are from Venus. Forbes. Nov 20, 2020. Crawford, J. & Lumpkin, J. R. (1983). The choice of selling as a career. Industrial Marketing Management. DOI:10.1016/S0019-8501(83)80005-4 Cummins, S., Peltier, J.W., Erffmeyer, R., & Whalen, J. (2013). A critical review of the literature for sales educators. Marketing & Entrepreneurship Faculty Publications. 5. Cummins, S., Peltier, J. W., Pomirleanu, N., Cross, J., & Simon, R. (2015). Evaluating educational practices for positively affecting student perceptions of a sales career. Marketing & Entrepreneurship Faculty Publications. 7. Deeter-Schmelz, D. R., & Norman Kennedy, K. (2011). The Journal of Personal Selling and Sales Management, Winter 2011, Vol. 31, No. 1, pp. 55-75 Donoho, C., & Heinze, T. (2011). The personal selling ethics scale: Revisions and expansions for teaching sales ethics. Journal of Marketing Education, 33(1) 107-122. 114 Emsi. (2020). Labor market data to help your institution, organization, or community thrive. Economicmodeling.com Fogel, S., Hoffmeister, D., Rocco, R. & Strunk, D. (2012). Teaching Sales, Harvard Business Review, 90, 94-99. Forbes, L., Loe, T., Patterson, R., & Erffmeyer, R. (2014). Establishing, growing, and running a sales program: An analysis of certified university sales centers, Journal of Selling, 14, 619. Gafford, J. M. (2019). Profit-Sharing vs Gainsharing: What drives millennials to a career in professional selling? Dissertation Manuscript. North Central University, School of Business and Technology Management. San Diego, CA George, B. (2018). Choosing the Right Kind of Accreditation for a Business School: A Comparison between AACSB, ACBSP, and IACBE. Journal of Research in Higher Education 2(2):45-61 DOI: 10.24193/JRHE.2018.2.3 Healy, W. J., Taran, Z., & Betts, S. C. (2011). Sales course design using experiential learning principles and Blooms taxonomy. Journal of Instructional Pedagogies, 6. Heifetz, R. A., Grashow, A., & Linsky, M. (2009). The practice of adaptive leadership: Tools and tactics for changing your organization and the world. Harvard Business Press. HR Daily Advisor Editorial Staff. (2018, September 19). Skilled sales talent is hard to come by, expensive to replace. HR Daily Advisor. https://hrdailyadvisor.blr.com/2018/09/19/skilled-sales-talent-is-hard-to-come-byexpensive-to-replace/ 115 Inks, S. A. & Avila, R. A. (2008). Preparing the next generation of sales professionals through social, experiential, and immersive learning experiences. Journal for Advancement of Marketing Education, 13(4), 47-55. Jelinek, R. (2018). Integrating SFA technology into the sales curriculum: Helping students understand what, why, and when. Marketing Education Review.Summer2018, Vol. 28 Issue 2, p80-88. DOI: 10.1080/10528008.2018.1464397 Job Placement Statistics (2019). Internal University Metrics. [Author Redacted]. Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112-133. Karakaya, F., Quigley, C. & Bingham, F. (2011). A cross-national investigation of student intentions to pursue a sales career. Journal of Marketing Education. https://doi.org/10.1177/0273475310389151 Kellerman, B. J. & Hekmat, F. (1989). Personal selling and sales management in the marketing curriculum: A status report. The Journal of Personal Selling and Sales Management, 9 (3) 35-45. Kirkpatrick, D. (1996). Revisiting Kirkpatricks four-level-model. Training & Development, 1, 54-57. Kirkpatrick, D. L., & Kirkpatrick, J. D. (2005). Transferring learning to behavior: using the four levels to improve performance (1st ed.). Berrett-Koehler. Retrieved from https://ebookcentral.proquest.com/lib/marianuin-ebooks/reader.action?docID=335397 Kunkle, M. & Peterson, R. (2018). Implications for business to business selling in the age of artificial intelligence. Sales Education Foundation. pp 1-8. Kurt, S. (2016, October 24). Kirkpatrick Model: Four Levels of Learning Evaluation. 116 Educational Technology. https://educationaltechnology.net/kirkpatrick-model-four-levelslearning-evaluation/ Lakhal, S. & Sevigny, S. (2015). The AACSB assurance of learning process: An assessment of current practices within the perspective of the unified view of validity. International Journal of Management Education, 13, 1-10. Lilly, B. & Stanley, S. M. (2016). Leveraging partnerships with local companies as a method of teaching sales: A constructivist approach. Journal for Advancement of Marketing Education, 24. Loe, T. W. & Inks, S. A. (2014). The advanced course in professional selling. Journal of Marketing Education, 36: 182. Manpower Group. (2018). Solving the talent shortage: Build, buy, borrow and bridge. 2018 Talent Shortage Survey. https://go.manpowergroup.com/talent-shortage-2018 Marshall, G. W. & Michaels, R. E. (2001). Teaching selling and sales management in the next millennium: An agenda from the AMA faculty consortium. Marketing Education Review, 11 (1). McConnell, C., Hoover, G., & Miller, G. (2008). Course embedded assessment and assurance of learning: Examples in business disciplines. Academy of Educational Leadership Journal, 12(3), 19-34. Moncrief, W. C., & Marshall, G. W. (2005). The evolution of the seven steps of selling, Industrial Marketing Management, (34)1, pp 13-22. Mullen, L. G., Larson, L. R. (2016). Analyzing the impact of a sales shadowing program: Process and outcomes. Journal for Advancement of Marketing Education, 24. 117 National Association of Colleges and Employers (NACE). (2019). First destinations for the college class of 2019. https://www.naceweb.org/job-market/graduate-outcomes/firstdestination/class-of-2019/interactive-dashboard/ National Association of Colleges and Employers (NACE). (2020). First destinations for the college class of 2020. https://www.naceweb.org/job-market/graduate-outcomes/firstdestination/class-of-2020/interactive-dashboard/ National Collegiate Sales Competition (NCSC). (2020). About us. https://www.ncscksu.org/about-us National Science Foundation. (1997). Analyzing Qualitative Data. User Friendly Handbook for Mixed Methods Evaluations. Ch. 4. https://www.nsf.gov/pubs/1997/nsf97153/start.htm Newberry, R. & Collins, M. K. (2015). Aligning sales curriculum content and pedagogy with practitioners needs. Research in Higher Education Journal, 27. 1-14. Nielson, B., & Cummins, S. (2019). Recruiting sales students: The value of professionals in the classroom. Marketing Education Review, 29(1), 65-74. Nietzel, M. T. (2020). Three options for reforming college faculty tenure. Forbes. Dec. 6, 2020. Office of Institutional Research (2020). Project Real Cost Financial Metrics. [Author Redacted]. Parker, R.S., Pettijohn, C.E. & Luke, R.H. (1996) Sales Representatives and Sales Professors: A Comparative Analysis of Sales Training Perceptions, Topics and Pedagogy, Marketing Education Review, 6:3, 41-50, DOI: 10.1080/10528008.1996.11488557 Peltier, J. W., Cummins, S., Pomirleanu, N., Cross, J., & Simon, R. (2014). A parsimonious instrument for predicting students intent to pursue a sales career: Scale development and validation. Journal of Marketing Education, 36, 62-74. 118 Pink, D. H. (2012). To sell is human: The surprising truth about moving others. New York, NY: Penguin Books. Praslova, L. (2010). Adaptation of Kirkpatricks four level model of training criteria to assessment of learning outcomes and program evaluation in Higher Education. Educational Assessment, Evaluation, and Accountability, 22, 215-225. Rodriguez, M., Honeycutt, E. D., & Ragland, C. (2015). Preliminary investigation of entertainment strategies involving alcohol: Implications for professional sales education and training in business markets. Journal of Business-to-Business Marketing. Oct-Dec2015, Vol. 22 Issue 4, p257-268. https://doi.org/10.1080/1051712X.2015.1115700 Sales Education Foundation. (2017). Building a university sales program: An opportunity to expand sales at the university level. Sales Education Foundation. https://salesfoundation.org/wp-content/uploads/2017/09/Building-A-University-SalesProgram.pdf Sales Education Foundation. (2019). Successfully activating corporate partnerships. Sales Education Foundation. https://salesfoundation.org/wpcontent/uploads/2019/11/Successfully-Activating-Corporate.2019-Annual.pdf Sales Education Foundation. (2020). Key statistics about university sales education. Sales Education Foundation. https://salesfoundation.org/wp-content/uploads/2020/06/2020Key-Statistics-Sales-education.pdf Sales Education Foundation. (2020). SEF Annual Magazine 2020. Sales Education Foundation. https://salesfoundation.org/SEFAnnual/ Sellars, D. (2005). Developing and Rote Playing Effective Sales Presentations, Canada: South Western. 119 Singh, J., Flaherty, K., Sohi, R., Deeter-Schmelz, D., Habel, J., Le Meunier-FitzHugh, K., Malshe, A., Mullins, R., & Onyemah, V. (2019). Sales profession and professionals in the age of digitization and artificial intelligence technologies: concepts, priorities, and questions. Journal of Personal Selling & Sales Management, 39 (1), 2-22. Spillan, J. E., Totten, J. W., & Chaubey, M. D. (2011). Exploring personal selling as a career option: A case study of the perceptions of African-American students. Academy of Marketing Studies Journal, suppl. Special Issue 2; Arden Vol. 15, 93-106. Spillan, J. E., Totten, J. W., & Ziemnowicz, C. (2007). What are students perceptions of a personal selling as a career? Journal for Advancement of Marketing Education, 11, 19-30 Spiller, L. D., Kim, D., & Aitken, K. (2019). Sales education in the United States: perspectives on curriculum and teaching practice, Journal of Marketing Education, Volume: 42, (3), 217-232. https://doi.org/10.1177/0273475319852756 Tan, K., & Newman, E. (2013). The evaluation of sales force training in retail organizations: a test of Kirkpatrick's four-level model. International Journal of Management, 30(2). p. 692. Accessed 27 Jan. 2021. Training Industry. (2020) Glossary Items. https://trainingindustry.com/glossary/knowledgeskills-and-abilities-ksa/ University Sales Center Alliance. (2020). Home. https://www.universitysalescenteralliance.org/content.aspx?page_id=0&club_id=554502 US News and World Report (2020). Sales representative overview. What is a sales representative? Money. (https://money.usnews.com/careers/best-jobs/sales-representative 120 Weilbaker, D. C. (2004). What to consider before developing a specialization (track, emphasis or center): personal selling as a template. Journal for Advancement of Marketing Education, 4 (Summer), 68-73. Williams June, A. (2020, November 12). Undergraduate-Enrollment Picture Worsens as Pandemic Drags On. Chronicle of Higher Education. https://www.chronicle.com/article/undergraduate-enrollment-picture-worsens-aspandemic-drags-on Wood, J. (2020). Indications that a course in sales leadership enhances early sales career success. Marketing Education Review, 30(3), 150 158. https://doi.org/10.1080/10528008.2020.1716805 121 Appendix A USCA Criteria for Membership 1. Become an Associate Member of the USCA for at least 12 months in good standing and meet all requirements for Associate membership. 2. Establish a university recognized stand-alone entity that will serve as the sales center. The entity must have a name and that name must have the word sales in some form (e.g., Sales Center; Selling Institute). 3. The sales center must have a budget. It must be able to receive funding, and the entity Director must have discretion to use funding and manage the budget. 4. Submit a document to the USCA President or Membership Chairperson which verifies the following for your school: a. Meeting the criteria outlined in #1, #2, and #3 above. b. Employs a minimum of 3 sales faculty (faculty must teach within sales curriculum), where 1 professor holds a full-time contract and a doctorate degree c. Operates a dedicated role play facility with recording capability d. Maintains a maximum of 30 students in entry-level sales classes, and a maximum of 20 students in advanced sales classes e. Provides 2 levels of sales training (e.g., basic and advanced) f. Offers a minimum of 3 sales courses 5. Full member status can be obtained without meeting one criteria if the University can demonstrate substantially exceeding requirements for other criteria 6. Pass a site visit Verification The President or other member of the USCA executive board shall conduct a site visit to verify status of program in accordance with the stated criteria for Full Membership. The site visit will include interviews with the Dean of the School and the Chair of the Department in which the Center is housed. 7. Attend the fall meeting each year Any full or associate member not attending the fall meeting will receive a letter of censure from the President of the USCA. Any full or associate member not attending two years in a row will be moved to suspended status (the University will be removed from USCA website). 122 Appendix B Research Instruments Qualitative Interview Guide Interview Guide: Sales Center Faculty/Sales Partners/Sales Research Organization Introduction Introduce self, my role, and purpose of the discussion. Review interview time (45 min to 1 hr). Restate information from invitation that interview is not recorded, that I will take notes, and that I will be documenting findings by organization type and job title, but not name of person. (Remind about my goals for this discussion) Role and Current Interactions with Partners Tell me a little bit about your role at ____________ and how you interact with partners (other universities or businesses). Probes: Interface with faculty, with students, others? Probes: Internship recruitment, FT job recruitment, speaking to classes/visits. Do you do all the recruiting or are there others with this responsibility? Do you use the college recruiting office or are you more likely to go directly to professors in the discipline from which you typically hire? Do you use any third party testing to assess potential candidates and if yes, who? What works well with that process today? Do you have any thoughts on how to improve your interactions with ________(faculty, students, career services, partners, etc.)? Probes: any partnerships where things work very well? What do they do? Any examples of what not to do? Probe: What is important when evaluating a potential university partnership? Knowledge and Skills needed by Graduates When thinking about what is required for your graduate/interns/new hires, what are the top skills that come to mind? Probes: Any others? If not mentioned, ask, What about (asking questions, listening, professionalism, presentations, telephone, time management, buying process, negotiation)? 123 Is there other knowledge that you believe is critical for new graduates to have, or are you seeing any new knowledge or skills starting to emerge or gaps in knowledge that are becoming more critical? Probe: If not mentioned, What about: CRM experience, AI, Virtual Selling, Sales Technology, Ethics Probe: Do you have any plans to include this/any of these in your own (curriculum/sales training/recommendations)? Businesses/Partners Only (Skip this section for faculty and other experts) Is there anything you would like to share about partnerships with universities? Do you have any advice, specifically to (university name) and our new sales program, about how we can best meet the needs of partners? Thank and end for Businesses/Partners Interview. Curriculum and Pedagogy Can you tell me a little about your curriculum and your suggestions for what courses you suggest for a new program? Probe: levels of classes, pre-reqs. What are your thoughts on curriculum needed for a new program that is just starting and has only a small number of students? Where would you focus your time? Which authors if any do you use to teach the selling process to your entry level sales employees? (e.g Spin Selling Neil Rackham) Is your sales curriculum open to non-business students? Why or why not? I know that role plays are commonly used in sales programs. Do you have any thoughts on that or any suggestions on other pedagogies that you have found to work well? Probes: What pedagogies have you found to be of most value? Why is that? Do you have any sources that I might reference to help me better understand those methods or how they are evaluated? Managing a Sales Program Id like to talk a little bit about what it takes to start and manage a sales program. Can you tell me how your program or typical programs are managed, especially when just getting started? 124 Probes: At what point (size of program/number of students/etc.) is a program director needed? Is that person responsible for fundraising? What other duties does that person have? What can you tell me about the fundraising process? How do you interface with your university fundraising group? What role, if any, does the Dean play in fundraising? (contacting partners, follow up, receipt and documentation of funds, budget authority, etc.). Anyone else involved? What suggestions do you have for a new program that is looking for partners? Probe: At what point do you ask for financial commitments? Do you typical ask for multi-year commitments? What level ($) of support would you feel a new, small program might achieve? Wrap up Do you have any other advice on ways to manage and grow the sales program, or on anything else weve discussed today? Thank and end Survey Instrument for Sales Students Thank you for participating in our student sales education survey. The results will be used as part of our sales program development and evaluation process. The survey should take about 5 minutes and is only for students who have completed at least one course in sales while in the (school name). No personally identifying information is being collected as part of this survey and your responses are completely anonymous. Please check all of the sales courses you have completed, including any course you are currently taking, while in the school. Which of the following types of sales competitions did you participate in? (Select all that apply) If you have not participated in any sales competitions, please select "None of these." Based on your experiences with the sales courses you have completed or are currently taking, please indicate your level of agreement with the following statements, from Strongly Agree to Strongly Disagree for each skill used in the selling process. The sales course(s) helped me build selling skills in: (list of skills) Based on your experiences with the sales courses you have completed or are currently taking, please indicate your level of agreement with the following statements, from Strongly Agree to Strongly Disagree for the knowledge and skills used in Self-Management and Customer Relationship Management. The sales course(s) helped me build knowledge and skills in: (list of skills) 125 Please indicate your level of agreement with each statement below from Strongly Agree to Strongly Disagree. I understand sales concepts and how to apply them I understand the sales process I know how to structure a sales presentation Have you previously or are you now completing an internship where at least part of your responsibilities are related to sales? What is your status in the school? I am a current student who will graduate in 2021 or earlier I am a current student who will graduate in 2022 or later What is your gender? Thank you for completing the survey. You may use the space below for any comments you wish to share about the sales program. 126 Appendix C Rubrics Level 1: Reaction, Student Evaluation of Instruction Questions (5-point scale, from Strongly Agree to Strongly Disagree) The syllabus clearly communicated the learning outcomes of the course. The readings, discussions, lectures, labs, and/or projects helped me attain the stated learning outcomes of this course. Multiple instructional methods were used in the course (e.g. lectures, problem solving, case studies, hands-on-activities, experiments, discussions, etc.). The instructional activities and assignments supported the course learning outcomes. The activities and assignments challenged me to think more deeply/critically about the course subject matter. Overall, (the Universitys) values of peace and justice, responsible stewardship, dignity of the individual, and reconciliation were reflected in the class. I would recommend this course to another student. The instructor demonstrated mastery of the subject matter. The instructor provided well-organized learning activities. The instructor provided prompt useful feedback. The instructor was available on a regular basis. The instructors behavior clearly reflected his/her genuine concern for my learning outcomes. I would recommend this instructor to another student. 127 Level 2: Learning, Role Play Rubric Student Name:____________________________________________ Concept/ Skills Approach Sales Process Professionalism Rapport Building Role Play Name: Ratings Pts 10 - 9 pts Mastery; great use 8 7 pts Partial Mastery; Used well but still room for improvement 6 pts Emerging; Gave it a good shot but not efficient yet 5 pts Still Developing; Attempted but felt weak 4 - 1 pts Tried but use was very limited at best 0 pts Not used at all 10 pts Mastery, great use 8 pts Used well but still room for improvement 6 pts Gave it a good shot but not efficient yet 4 pts Attempted but felt weak 2 pts Tried but use was very limited at best 0 pts Not used at all Effective Questioning Questioning 10 pts Mastery, great use 8 pts Used well but still room for improvement 6 pts Gave it a good shot but not efficient yet 4 pts Attempted but felt weak 2 pts Tried but use was very limited at best 0 pts Not used at all Product Presentation Communication/ Speaking 10 pts Mastery, great use 8 pts Used well but still room for improvement 6 pts Gave it a good shot but not efficient yet 4 pts Attempted but felt weak 2 pts Tried but use was very limited at best 0 pts Not used at all 10 pts Mastery, great use 8 pts Used well but still room for improvement 6 pts Gave it a good shot but not efficient yet 4 pts Attempted but felt weak 2 pts Tried but use was very limited at best 0 pts Not used at all Trust/Overcoming Objections Listening Building Trust/Ethics Obtain Commitment Persuasion/ Influence Solving Customer Problems Total Points: (out of 50) 128 Level 3: Behavior/Transfer, Negotiation Final Project Rubric Name___________________________ Date_______ Role Assigned_____________________ CONCEPT RATING 10 9 pts Mastery of the concept Prepared for the case Interests BATNA 8 7 pts Partial Mastery; Good attempts but room for improvement 6 pts Not efficient yet 5 pts Attempted but weak PTS 4 - 1 pts Very limited but some 0 pts Not used at all Grasped the issues of the case Asked questions to understand the interests of the other party Generated options to problem solve and grow the pie Behaved in ways to build trust Level 3: Behavior/Transfer, Competition Rubrics are not included due to the proprietary information they contain. 129 Level 3: Behavior/Transfer, Supervisor Internship Evaluation Final Internship Assessment By Internship Supervisor Student Name: _________________________________ Date: ________________ Intern Title: __________________________ Company Name: _____________ PART I: To be completed at the end of the students internship. Please provide your candid evaluation of this students performance or skill level in each of the following areas based on this internship experience. This evaluation is not confidential and will be shared with the student and (Study) University. 1=Lacks this skill, 2=Limited/minimal skill level, 3= Adequate/average skill level, 4=Above average skill level, 5=Exceptional skill level, N/A Not Applicable. Teamwork 1. Makes a positive impact on work team by establishing rapport and credibility. 2. Shares information/resources with others. 3. Assists/cooperates with co-workers. 4. Able to addresses business challenges in a team or collaborative setting. Communication Skills 1. Demonstrates oral communication skills required for the job. 2. Writes clearly and concisely. 3. Is willing to speak up, communicate information, and ask for clarification. 4. Listens to feedback and acts to improve. 5. Properly uses the specialized terms associated with the interns discipline. 130 Appendix D Display Matrix of Knowledge, Skills, and Pedagogy Findings Respondent Group Faculty at Other Schools Curriculum Recommendations Minimum two levels of sales courses, an introductory and an advanced; both should cover the sales process. Sales competitions are important so that students can apply sales concepts. Business Partners Sales Organization Supporting Statements USCA requires two levels and this is the only certifying organization for sales. Internships and competitions will help students with job placement. Sales technology continues to grow in importance, including virtual selling. Consider a technology course that Virtual selling is growing and students need to understand how to develop and includes presentation technologies/virtual presentations. deliver a good presentation in-person and remotely. Would like a dedicated sales This provides a good path for companies to course at the introductory level recruit students. where partners can interact with Introductory courses help students students. understand if sales is the right career Sales internships and choice for them. competitions are important; they Our other partner universities support sales expose students to real sales internships and/or competitions. scenarios. Minimum two levels of sales This is the recommendation from USCA. courses, an introductory and an Top programs have up to five sales advanced; more course are better. courses. Consider specialty areas in sales, This will attract more business partners and such as healthcare, finance, could help differentiate from other insurance. programs. Internships and competition Competitions allow students to apply what participation are needed. they have learned in the classroom. 131 Display Matrix of Knowledge, Skills, and Pedagogy Findings Respondent Group Faculty at Other Schools Business Partners Sales Organization Skills/Knowledge needed Listening Questioning Persuasion Networking Rapport building Sales Process Time Management Teamwork Solving Problems Sales Technology Sales Presentations Listening Speaking Writing Sales Process Networking Rapport Building Telephone/email etiquette/Social Persuasion Attitude/ Motivation Financial Analysis Questioning Listening Build rapport Interpreting/ Assessing Competitive Persuasive Detail oriented Pedagogy Suggestions Supporting Statements Videotaped Role Plays Competitions (internal and external) Class exercises Group Projects/ Presentations Sales pitches to class Sell something on campus Sales rep shadowing Online technology Involve business partners Mock interviews Pitches/Presentations Competitions Internships Partner interactions during class/outside of class Use skills assessment tools Use the Culture Index Have seen success in our programs (job placement, at competitions). Use skills assessment tools The Chally Assessment One on one coaching Competitions Classroom learning Role Plays USCA recommendation Graduate interview/survey data. Funding by industry from involving them in the program. Seen at other partner universities We use these tools with all new hires and they work to identify strengths and weaknesses. If you can teach them these skills, we can teach them about our specific industry/product. Most top programs use these tools. Research Publications Knowledge from other university sales programs 132 Course Framework for Knowledge and Skills Knowledge or Skill Listening Questioning Sales Process/TAO/SPIN Building Trust/Ethics Persuasion/Influence Communication/Speaking Professionalism Rapport Building Solving Customer Problems Networking Collaboration/Teamwork Sales Presentations Negotiation Time Management Interpreting Technology/Sales Tools CRM Software & Process Writing for Sales Financial Analysis/ Forecasting Introductory Course I, A I, A I, A I, A I, A I, A I I, A I, A Advanced Course R, A R, A R, A R, A R, A R, A R, A R, A R, A I, A I, A I, A I I, A I I I, A I I, A Sales Technology R R R I, A R, A I, A R, A R, A R, A I R, A Negotiation R, A R, A R, A R, A R R R, A R R, A R, A R, A Course Framework with Pedagogy Use Pedagogies Used Interaction with & Coaching from Sales Professionals Guest Speakers Role Plays Presentations Exams/Quizzes Case Studies Lecture Competitions - Regional/National Competitions - Internal/Local Resources, e.g., videos, online examples Class Projects Assignments/Writing Assignments Peer Coaching In-field job shadowing Introductory Course Advanced Course Sales Negotiation Technology ...