Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
Hawaii Statewide Evaluation of the 21st Century Community Learning Centers Program: School Year 2018-19
Year Published: 2020
A statewide evaluation of Hawaii’s 21st Century Community Learning Center (21st CCLC) programs during the 2018-19 program year found students in the program made academic and behavioral gains. The percent of students who missed 15 or more days of school throughout the year was significantly less for 21st CCLC participants compared to non-participants. In addition, those who participated in 21st CCLC programs were more likely to meet or exceed proficiency in math and reading compared to non-participants.
Program Name: Hawaii 21st Century Community Learning Centers
Program Description:
Hawaii’s 21st Century Community Learning Center program, which receives federal funding through the 21st CCLC Initiative, provides academic enrichment opportunities for children at high-poverty and low-performing schools. During the 2018-19 program year, 25 grantees operated 92 sites, serving 14,745 youth.
Scope of the Evaluation: Statewide
Program Type: Afterschool
Location: Hawaii
Grade level: Elementary School, Middle School, High School
Program Demographics:
A majority of schools served by Hawaii’s 21st CCLC programs are Title I eligible, meaning more than 40 percent of students qualify for the Free and Reduced Price Lunch program (FRPL). During the 2018-19 program year, 51 percent of students were eligible for FRPL, 9 percent were identified as students with special needs, and 8 percent were identified as English Language Learners. Regarding race and ethnicity, 36 percent identified as Native Hawaiian/ Pacific Islander, 23 percent as Asian, 17 percent as two or more races, 14 percent as Hispanic/Latino, 7 percent as White, and 1 percent as Black/African American.
Program Website: https://www.hawaiipublicschools.org/ParentsAndStudents/SupportForStudents/21stCCLC/Pages/default.aspx
Evaluator: Barker, L.T. & McLelland, C. IMPAQ International, LLC.
Evaluation Methods:
This evaluation uses data from four primary sources: the Annual Performance Report (APR) submitted by subgrantees, subgrantee evaluation reports, output reports from the Hawaii Department of Education’s longitudinal data system, and data stories that include data on students at participating schools who did not participate in 21st CCLC. This is the first year that the Hawaii Department of Education is reporting behavioral and academic outcomes based on the state database rather than teacher surveys.
Evaluation Type: Quasi-experimental
Summary of Outcomes:
A statewide evaluation of Hawaii’s 21st CCLC programs found that students regularly participating in 21st CCLC programs (30 days or more) made significant gains in behavior and academics compared to those who did not participate. For those who participated in 21st CCLC programs for at least 30 days, only 12.6 percent of students were chronically absent (missed more than 15 days) compared to 24 percent of non-participants. The differences were statistically significant.
In regards to academic achievement, students participating in 21st CCLC programs were more likely to meet or exceed proficiency in English Language Arts (ELA) compared to non-participants (49 percent vs. 42.1 percent). Those who participated in 21st CCLC programs for at least 30 days were even more likely to be proficient (51.7 percent). The report found that 1,596 21st CCLC students improved in English, including those who did not need to improve.
Similar results were found for math achievement. 21st CCLC participants were more likely to meet or exceed proficiency in math compared to those who did not attend (35.8 percent vs. 30.9 percent), with the highest numbers reported among those who participated in 21st CCLC programs for at least 30 days (40.9 percent). Overall, 1,245 21st CCLC students improved in math, including those who did not need to improve.
Regarding proficiency in science, the report found that 21st CCLC students were more likely to meet or exceed proficiency in science compared to non-participants (20.1 percent vs. 16.3 percent). Similar to the results in ELA and math, students who participated in 21st CCLC programs for at least 30 days were the most likely to meet or exceed proficiency in science (27.7 percent).
The evaluation also looked at performance indicators for serving communities most in need, services to parents and families, community partnerships, and hours of operation.
Compared to statewide numbers, 21st CCLC programs served a higher percentage of students who qualify for FRPL (48.9 percent vs. 50.9 percent) and were more likely to be located at a Title I school (65.1 percent vs. 67.4 percent). Authors concluded that their findings suggest that, “the 21st CCLC program effectively targeted schools and communities with the greatest need for the program’s services.”
Regarding family engagement, most centers encouraged family engagement, serving 11,967 family members. There was a continual increase from the 2014-15 school year to 2018-19, with a large jump from 7,338 family members in 2017-18 to more than 11,900 in 2018-19.
Additional areas of growth were programs’ hours of operation and partnerships. The percent of centers offering at least 12 hours a week increased from the 2018-19 school year, reaching 71 percent, and subgrantees worked with an average of 29 community partners, with a significant growth in number of partnerships from 63 overall during the 2014-15 school year to 739 in 2018-19 school year.
Date Added: June 17, 2021
