MoHealthWINS DOL Grant Federal Investment = $19,982,296 Executive Summary 

● Enrollment of 4,251 surpassed grant target by 19%. 

● Colleges used employer input and engagement to create and/or redesign nearly 60 programs in nearly 40 programmatic areas built upon industry-requested stackable credentials. 

● Colleges targeted and provided college access to unemployed and academically low-skilled adults: 

o Average age of participants was 35. 

o 2% were TAA eligible. 

o 82% were either unemployed or under-employed at program start-up. o 75% were academically low-skilled at program start-up. 

● Throughout the grant, colleges developed and improved relationships with local career centers. Nearly 1,200 of the participants (28%) were referred to a campus by a Career Center. 

● The credit hour completed to attempted ratio for the credit programs was 88%, while the credit hour completed to attempted ratio for non-credit programs was 70%. ● More than 2,800 (n = 2,812) of the participants completed at least one program of study generating a program completion rate of 66%. Counting all program awards and stackable credentials, this group of completers were awarded 10,998 industry-requested awards/credentials. 

● Eighty percent of the program completers secured employment upon program completion, with an annual average wage of $22,540. 

● Seventy-five percent of the program completers who started as unemployed secured employment upon program completion with an annual average wage of $23,050. ● Grant participants expressed a high degree of satisfaction with program offerings and college support services. In addition, participants reported grant programs/services helped to improve their abilities and self-efficacy with regard to key workplace skills. ● Through the development and implementation of short-term, career programs, MHW grant participants were more likely than non-grant students to complete a program award. ● Individual campus culture/climate certainly impacted the extent to which MHW innovations and experimentation were supported. For those campuses who embraced the experimental nature of MHW, the grant has laid a solid foundation for further innovations associated with the following areas: development and redesign of programs using career pathways; re-design of developmental education; adoption of intrusive student and instructional support strategies; and expanded use of employer engagement to support program creation and continuous improvement. 

MoMANWINS DOL Grant Federal Investment = $14,900,000

Executive Summary 

This report presents an evaluation of Missouri’s Manufacturing Workforce Innovation Networks: MoManufacturingWINs (commonly referred to as MMW) grant as part of the Round 2 Trade Adjustment Assistance Community College and Career Training (TAACCCT) program administered by the U.S. Department of Labor (DOL). This is the culminating report on grant implementation, performance outcomes, and impact. Evaluation progress reports completed during the grant and the curriculum review report are attached. 

In 2012, Missouri received a $14.9 million Round Two TAACCCT grant to meet the State’s growing demand for manufacturing related workers. Eight Missouri community colleges and the State’s technical college came together under the Missouri Community College Association (MCCA) leadership to form the MMW consortium. Missouri community colleges operate as a decentralized system, thus MCCA was asked to lead and administer the MMW grant. 

The MMW consortium colleges recognized the imperative to improve their instructional programs and support services to better meet the needs of adult learners and other grant target populations, including TAA-eligible and Veteran students. To serve and impact these populations, the colleges needed to engage employers to help design new or enhance existing programs of study based upon industry-recognized, stackable credentials to align with existing or emerging manufacturing competencies and career steps. The colleges also understood the need to develop programs of study which could be completed in a condensed/accelerated manner and ultimately lead to employment in the manufacturing sector. 

To support the proposed theory of change, colleges employed a stackable certificate model connecting program awards to appropriate industry certifications and providing a map for completion based on “Manufacturing Career Pathways”. This model allowed for multiple 

entrance and exit points to give students options for training in short increments. To further support this model, colleges developed a learning framework offering students basic academic skills through contextualized courses as well as intrusive support services to help students prepare for and persevere to completion and employment. 

This report provides both implementation and grant performance outcome evaluation, and it examines the impact of grant programs and strategies with regard to program completion and employment upon program completion. A multi-dimensional evaluation process was employed to meet Department of Labor evaluation requirements and provide the consortium and its member colleges with data and analysis related to the following key questions outlined below and referenced throughout this report. 

The following represent key data results related to this evaluation. 

● Enrollment of 4,547 surpassed grant target by 37%. 

● Grant Program of Study (POS) completers (3,295) surpassed the grant target by 86%. The Program of Study completion rate of 72% surpassed the grant target program completion rate of 53%.

● Grant Program of Study completers employed at program completion (3,033) surpassed the grant target of program completers employed by 111%. The employment rate for grant POS completers of 92% surpassed the grant target employment rate of 81%. 

● Colleges used employer input and engagement to create and/or redesign 44 programs built upon industry-requested stackable credentials. 

● Colleges provided college access to unemployed and academically low-skilled adults and other key target groups. 

▪ Average age of participants was 35. 

▪ 2% were TAA eligible. 

▪ 12% were Veterans. 

▪ 86% were either unemployed or under-employed at program start-up. 

▪ 78% were academically low-skilled at program start-up. 

▪ 54% were enrolling in college for the first-time 

● Throughout the grant, colleges partnered with local Career Centers to recruit students. Fourteen percent of the participants (659) were referred to a campus by a Career Center. 

● The credit hour-completed-to-attempted ratio for the credit programs was 92%, while the credit hour-completed-to-attempted ratio for non-credit programs was 70%. 

● A total of 3,295 participants completed at least one program of study generating a program completion rate of 72%. Counting all program awards and stackable credentials, this group of completers received 8,873 industry-requested awards/credentials. 

● Colleges awarded 4,599 awards/certifications recognized by the following professional societies: Manufacturing Skills Standards Council; Society of Manufacturing Engineers; American Welding Society; and National Institute for Metalworking Skills. 

● Ninety-two percent of the program completers secured employment upon program completion with an annual average wage of $56,000. 

● Eighty-six percent of the program completers who started as unemployed secured employment upon program completion with an annual average wage of $43,000. 

● Grant participants expressed a high degree of satisfaction with program offerings and college support services. In addition, participants reported grant programs/services helped to improve their abilities and self-efficacy with regard to essential workplace skills. 

● Through the development and implementation of short-term, career programs, MMW grant participants were more likely than non-grant students to complete a program award. Data presented in this report point to the success of MMW grant participants and reveal that grant participants completed programs and secured employment at higher rates than students in more traditional, non-grant programs. Although such results are encouraging, it is important for the consortium to focus on lessons learned during the grant. These lessons include the following: the importance of connecting classroom faculty, advisors, and instructional support staff; accelerated programs and curriculum often require increased instructional support for students; advising and career coaching is a continuous process that covers the entire student experience from recruitment to program completion and onto employment; programs connected to career

pathways and built upon industry-recognized credentials are valued by students and employers; accelerated and contextualized approaches to developmental education provide meaningful alternatives to more traditional, term-based developmental education models; and community and employer partnerships must be continuously cultivated in order to produce intended results. 

Individual campus culture/climate certainly impacted the extent to which MMW innovations and experimentation were supported. For those campuses who embraced the experimental nature of MMW, the grant has laid a solid foundation for further development, scaling and sustainability of efforts associated with the following areas: 

MoManufacturingWINs continued the state’s experience in working together as a consortium and, by the end of the grant, the colleges found that the consortium framework worked for them. Colleges consistently reported to the TPE the benefits of working and learning together as they implemented grant programs and strategies. College faculty and staff found the connections made to be useful in carrying out consortium-specific work and also expanding such efforts to other non-grant areas. One long-time college leader stated that the MoWINs consortia fundamentally changed the way the colleges think about how to approach new initiatives. Prior to Missouri’s TAACCCT grants, such cooperation and sharing among the colleges was not as prevalent. 

The Missouri Community College Association’s new Executive Director recognizes this increased level of statewide cooperation and sees it as an opportunity for transformative change in how the State’s community colleges respond to student, community and employers’ needs. To support such transformative change, MCCA has incorporated the lessons learned from its Round 1 and Round 2 TAACCCT grant into its current strategic planning process.

MoSTEMWINS DOL Grant Federal Investment = $14,900,000 Executive Summary 

This report presents an evaluation of Missouri’s STEM Workforce Innovation Networks: MoSTEMWINs (MoSTEM or MSW) grant as part of the Round 4 Trade Adjustment Assistance Community College and Career Training (TAACCCT) program administered by the U.S. Department of Labor (DOL). This is the culminating report on grant implementation, performance outcomes, and impact. 

In 2014, Missouri received a $14.9 million Round Four TAACCCT grant to meet the State’s growing demand for STEM related workers. Twelve Missouri community colleges and the State’s technical college came together under Missouri Community College Association (MCCA) leadership to form the MSW consortium. Missouri community colleges operate as a decentralized system, thus MCCA partnered with the grant’s lead college (Metropolitan Community College) to coordinate and administer the MSW grant. 

The MSW consortium colleges recognized the imperative to improve their instructional programs and support services to better meet the needs of adult-learners and other grant target populations, including TAA-eligible and Veteran students. To serve and impact these populations, the colleges needed to engage employers to help design new or enhance existing programs of study based upon industry-recognized, stackable credentials to align with existing or emerging STEM-related workforce pipeline needs. The colleges also understood the need to develop programs of study which could be completed in a condensed/accelerated manner and ultimately lead to employment in the following occupational clusters: Information Technology, Health Sciences, Life Sciences, Manufacturing, and Transportation. 

To support the proposed theory of change, colleges employed a stackable certificate model connecting program awards to appropriate industry certifications associated with STEM occupations and careers. This model allowed for multiple entrance and exit points to give students options for training in short increments. To further support this model, colleges developed a learning framework offering students 

basic academic skills through contextualized courses as well as intrusive support services to help students prepare for and persevere to completion and employment. 

The following represent key data results related to this evaluation. 

● Enrollment of 2,935 surpassed grant target by 58%. 

● Grant Program of Study (POS) completers (1,903) surpassed the grant target by 28%. The Program of Study completion rate of 65% was less than the grant target program completion rate of 80%. 

● Grant Program of Study (POS) completers employed at program completion (1,141) surpassed the grant target of program completers employed by 54%. The employment rate for grant POS completers of 60% surpassed the grant target employment rate of 50%. 

● Colleges used employer input and engagement to create and/or redesign 40 programs built upon industry-requested stackable credentials. 

● Colleges provided college access to unemployed and academically low-skilled adults and other key target groups. 

▪ Average age of participants was 33. 

▪ 2% were TAA eligible. 

▪ 7% were Veterans.

▪ 89% were either unemployed or under-employed at program start-up. 

▪ 60% were academically low-skilled at program start-up. 

▪ 41% were enrolling in college for the first-time 

● Throughout the grant, colleges partnered with local Career Centers and local employers/community partners to recruit students. Eleven percent of the participants (325) were referred to a campus by a Career Center and 20% (574) were referred by a local employer or other community partner. 

● A total of 1,903 participants completed at least one program of study. Counting all program awards and stackable credentials, participants received 3,935 industry-requested awards/credentials. 

● Sixty percent of the program completers secured employment upon program completion with an annual estimated average wage of $28,000. Due to incomplete wage data, the TPE can only estimate an annual average wage based upon available post-grant quarterly earnings. 

● Fifty-eight percent of the program completers who started as unemployed secured employment upon program completion with an annual estimated average wage of $27,000. Due to incomplete wage data, the TPE can only estimate an annual average wage based upon available post-grant quarterly earnings. 

● Through the development and implementation of short-term, career programs, MSW grant participants appeared more likely than non-grant students to complete a program award. 

Data presented in this report point to the success of MSW grant participants and reveal that grant participants completed programs and secured employment at higher rates than students in more traditional, non-grant programs. Although such results are encouraging, it is important for the consortium to focus on lessons learned during the grant. These lessons include the following. 

● The importance of connecting classroom faculty, advisors, and instructional support staff as accelerated programs and curriculum often require increased instructional support for students. ● Advising and career coaching is a continuous process that covers the entire student experience from recruitment to program completion and onto employment. 

● Programs connected to career pathways and built upon industry-recognized credentials are valued by students and employers. 

● Accelerated and contextualized approaches to developmental education provide meaningful alternatives to more traditional, term-based developmental education models. 

● Community and employer partnerships must be continuously cultivated to produce intended results. ● When developing and implementing programs and instructional support strategies for the adult, first-time college segment, colleges recognized the importance of creating multiple points of intervention and support across the entire student experience from recruitment and program onboarding, and onto program retention and completion, and, finally, employment. In addition, college faculty and staff grew to understand it was their responsibility to pro-actively engage with students to help ensure students connected with such interventions and support services. To help further explore this phenomenon we operationalized this process as the F.O.R.C.E. model (Find/Recruit, Onboard to Career Pathway, Retain Students, Completion of Program, and Employed Post Program Completion). 

Individual campus culture/climate certainly impacted the extent to which MSW innovations and experimentation were supported. For those campuses who embraced the experimental nature of MSW, the grant has laid a solid foundation for further development, scaling and sustainability of efforts highlighted below.

In such instances, the TPE saw evidence of the following scientific inquiry process: identification of an area for investigation; development of a strategy and hypothesis for impact; collection and interpretation of data related to hypothesis; and use of data analysis for 

continuous improvement. 

It is worth noting 

MoSTEMWINs was 

Missouri’s third statewide 

TAACCCT grant and TPE 

observations point to the 

continued development of 

statewide lessons learned. 

The Round 4 

MoSTEMWINs grant was 

built upon accomplishments and lessons learned from Rounds 1 and 2. A key piece of this cumulative experience is connected to the state’s experience in working together as a consortium. The consortium helped develop and expand a learning network between and among partner colleges. Colleges consistently reported to the TPE the benefits of working and learning together as they implemented grant programs and strategies. College faculty and staff found the connections made to be useful in carrying out consortium-specific work and expanding such efforts to other non-grant areas. 

MoAMPWINs DOL Apprenticeship Grant Federal Investment = $12,500,000 Executive Summary 

Grant Vision 

The Scaling Apprenticeship through Sector-Based Strategies program was launched in 2019 by the United States Department of Labor (DOL). The DOL awarded $12.5 million to St. Louis Community College to manage the MoAMP statewide consortium. The primary goal for this DOL program was “to accelerate the expansion of apprenticeships to new industry sectors reliant on H-1B visas, such as information technology (IT) and IT-related industries, advanced manufacturing, and health care”. 

(https://www.apprenticeship.gov/investments-tax-credits-and-tuition-support/scaling-apprentices hip-through-sector-based-strategies.) The Missouri MoAMP consortium directed its attention to 

the advanced manufacturing sector and the following vision: engage with industry & external partners to improve member colleges’ ability to create and deliver innovative pre-apprenticeship, unregistered, and apprenticeship training programs resulting in skills, credentials and degrees connected to high wage, high-skill occupations in advanced manufacturing. 

Grant Implementation 

MoAMP member colleges and consortium leadership recognized the grant as an opportunity to implement and experiment with employer engagement strategies and apprenticeship models to help employers and

students see the value of such models. To this end, MoAMP partnered with its Third-Party Evaluator (TPE) to connect MoAMP efforts to Deming’s Plan, Do, Study & Act model (PDSA)1. This integrated learning & continuous improvement model allowed grant practitioners and the TPE to monitor implementation strategies and outcomes in real-time and explore signs of progress, success, challenges, and areas for improvement. 

As with previous DOL TAACCCT2 grants, when called upon to develop innovative instructional and student support models to meet Missouri’s workforce development needs, Missouri’s community colleges delivered. Evaluation data notes the consortium, and its member colleges addressed grant activities, deliverables, and milestones with fidelity and used ongoing evaluation results for continuous improvement throughout the grant cycle. MoAMP was a complex endeavor with nearly 600 individual activities, deliverables, and milestones (ADMs). Final implementation evaluation data reveals member colleges progressed to the Advancing, Mature and/or Sustaining Implementation stage for 90% of MoAMP ADMs. A key feature of such successful strategy implementation involved MoAMP colleges partnering with 433 employers to create, launch, and support a full spectrum of apprenticeship models (Registered, Unregistered, Industry Recognized, and Pre-Apprenticeship), to address employer talent demands and needs. 

Grant Performance 

As a result of the successful implementation of grant ADMs, the consortium and its member colleges exceeded nearly all DOL’s targeted performance outcomes, including enrolling 5,862 participants thus exceeding the target of 5,000. This accomplishment points to the persistent efforts of the colleges to build employer partnerships, customized apprenticeships, and earn and learn models to convey the return on value of apprenticeship programming to employers while meeting employer and student needs. A summary of key MoAMP grant performance outcomes is presented below. 

1 https://deming.org/explore/pdsa/

2 https://www.dol.gov/agencies/eta/skills-training-grants/community-colleges 

STUDENT VOICES 

TO LEARN MORE ABOUT HOW MOAMP TRAINING IMPACTED STUDENT LIVES, CLICK ON THE FOLLOWING LINKS. 

https://www.youtube.com/watch?v=a2ty9CTRibE 

Precision Machining Technology Program at STLCC (youtube.com) 

Boeing Sheet Metal Assembler & Riveter (SMAR) Pre-Employment Training at STLCC (youtube.com) 

General Maintenance Mechanic Training at STLCC (youtube.com) 

Boeing Composites Mechanic Pre-Employment Training at STLCC (youtube.com) https://mccamoamp.com/ 

LEARNING & SUSTAINABILITY 

BEFORE EXPLORING WHAT COLLEGES LEARNED AS THEY IMPLEMENTED GRANT STRATEGIES, LET’S REVISIT THE CONSORTIUM’S COMMITMENT TO experiment with employer engagement strategies and apprenticeship models to help employers and students see the value of such models. This commitment and the colleges’ willingness to use lessons-learned were especially important during the initial stages of the grant (2019-2020) when COVID-related challenges negatively impacted efforts to recruit employer partners and students to apprenticeship opportunities. 

The MoAMP college leads should be applauded for using their grant evaluation processes to continually track and record lessons-learned along their MoAMP journey. Despite the initial challenges associated with COVID and the difficulty of creating and maintaining employer engagement and student recruitment efforts during COVID-recovery efforts, MoAMP college leads continued to meet, and collectively review and share lessons learned. Evaluation data suggest that such collective problem solving played a significant role in helping MoAMP outperform its outcome measures. When considering data related to what colleges learned as they implemented grant strategies, the following overarching principles were identified by MoAMP campus leads. 

▪ Apprenticeship programming is not one size fits all. Instead, programs should be designed to align with employer culture, needs, and expectations as well as college resources. Clear program structure and employer responsibilities increases employers’ likelihood to engage in “apprenticeship” partnerships and programming. 

▪ It is imperative to succinctly articulate the ROV (Return on Value) of the apprenticeship to the employer even when employers are willing and eager to participate in apprenticeship training programs. Employers value the following concepts when considering ROV: talent pipeline development, onboarding of work-ready employees, cost savings related to “good hires vs poor hires;” and increased likelihood of retaining quality employees. 

▪ Despite accelerated pathways to industry-recognized training and appropriate college certificates/credentials, students need intentional support strategies to overcome

life-challenges related to program enrollment, retention, and completion. This is especially true for adult and traditionally underserved populations. 

▪ Program modeling and student recruitment based on Earn & Learn instructional strategies can help bridge the economic gap for currently unemployed and under-employed students who are seeking to improve their economic and wage outlook. 

▪ Onboarding students to career pathways using short-term Pre-Apprenticeship programs is effective in helping students begin to see the value and return on investment in participation in apprenticeship training. 

▪ College organizational and leadership commitment to and investment in support of apprenticeship programming, including strategies to connect such programming to college certificates/degrees is necessary to grow apprenticeships. 

▪ Statewide collaboration provides valuable opportunities for exploring potential grant/funding, information sharing and collective problem solving. MCCA has already taken steps to enhance such collaboration by creating a statewide grants council to review potential external funding sources beneficial to colleges’ and MCCA’s strategic directions. 

Moving forward, MoAMP member colleges have identified practices and innovations which hold promise and potential for a greater impact associated with apprenticeship programming. These practices and innovations are presented below. 

PROGRESS & POTENTIAL 

Connecting learn & earn models to career pathways, incorporating 

stackable credentials, embedding industry-recognized certifications within programs, offering non-traditional scheduling, connecting short-term, 

pre-apprenticeship programs to full apprenticeship model, and use of 

credit for prior learning.

Understanding barriers for under-served populations; recognizing 

accelerated programs for such students require support services; 

expanding college services to help students navigate and remain in 

training programs. 

Maintaining momentum and organizational support for new concepts/shifts 

in organizational culture; enhancing executive leadership support for and 

connection to innovation, experimentation, and expansion of CTE and 

apprenticeship models through workforce development partnerships. 

Sharing and expanding lessons learned regarding employer engagement 

and the need to move beyond traditional program advisory models; 

connecting employer-recognized credentials to hands-on training; 

educating employers about the Return on Value of apprenticeship; and 

clarifying apprenticeship models, processes, and expectations. 

Systematically and intentionally connecting promising strategies to 

continuous improvement processes and college strategic planning efforts. 

Demonstrating to students and employers the value of the Earn & Learn 

models contextualized to the workplace environment and supported with 

hands-on training. Clearly articulating to students how apprenticeship 

models connect to career pathway & stackable credentials & future 

earnings. 

In conclusion, it is the opinion of C&A that despite COVID-related challenges, the MoAMP consortium and its member colleges successfully partnered with employers to implement grant strategies with fidelity. Furthermore, C&A notes evidence MoAMP met or exceeded nearly all grant performance targets. This strong performance and the use of lessons-learned for continuous improvement have laid a foundation for further development of apprenticeship practices and models. 

MOAMP FINAL SUMMARY DOL REPORTED OUTCOMES 

outcome performance is presented below.

Total number of unemployed and underemployed apprentices prior to 

enrollment who complete an apprenticeship education/training program and 5 

337 

maintain their employment status with a current or new employer 

Incumbent Workers not included in this outcome. 

Total number of incumbent worker apprentices who complete an 

apprenticeship education/training program and advance into a new position. Includes incumbent workers who advanced into a new position with their current 6 

327 

employer or a new employer following the completion of a training program. 

Incumbent workers who did not advance into a new position (i.e., who retained their existing position or layoff aversion) following the completion of a training program should not be included in this outcome. 

B Expanding Apprenticeship Program OutputsMoAM P Total 

1Total number of newly created apprenticeship programs, including Registered Apprenticeship programs124 2Total number of employers engaged (i.e., those employers that adopt apprenticeship programs as a result of your grant project)150 Total number of expanded apprenticeship programs, including Registered 

3 

31

Apprenticeship (e.g., new industries, occupations, or service areas, or increasing the number of apprentices registered)