Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
Indiana 21st Century Community Learning Centers Statewide Evaluation Report: 2017 & 2018
Year Published: 2019
A statewide evaluation of Indiana’s 21st Century Community Learning Center (21st CCLC) programs uses comparative analysis to show that high-participating program participants are more likely to experience gains in academic performance and behavior when compared to students who participate in 21st CCLC programs less frequently. During the 2017-2018 program year, a higher percentage of participants who attended the program for 90+ days passed the English/language arts (52 percent), math (48 percent), and science (41 percent) state standardized tests compared to those attending less frequently. Teachers also reported that more than 6 out of 10 students attending 60 or more days and identified as needing to improve in the following respects, improved their academic performance (68 percent), classroom participation (64 percent), and quality of homework completion (63 percent).
Program Name: Indiana 21st Century Community Learning Centers
Program Description:
Indiana’s 21st Century Community Learning Center program—which receives federal support through the 21st CCLC Initiative—serves high-needs communities across the state, providing local afterschool and summer programming through 129 grantees in 46 counties operating 225 centers serving 23,928 students during the 2017-18 program year.
Scope of the Evaluation: Statewide
Program Type: Summer, Afterschool, Before school
Location: Indiana
Grade level: Elementary School, Middle School, High School
Program Demographics:
All schools served by Indiana’s 21st CCLC programs are Title I eligible, meaning more than 40 percent of students qualify for Free and Reduced Price Lunch. In 2017-2018 program year, 75 percent of students qualified for Federal Free or Reduced Price Lunch, 6 percent of students had limited English proficiency, and 9 percent of students had special needs. With regard to race and ethnicity, 53 percent of students identified as White and 47 percent identified as a “minority.”
Program Website: https://www.doe.in.gov/21stcclc
Evaluator: Diehl Consulting Group
Evaluation Methods:
This evaluation collected data through the Indiana Department of Education’s EZReports Afterschool Reporting System in order to conduct a descriptive and exploratory matched-group analysis of 21st CCLC participants. Teacher survey information, student grades, and state assessment test scores were used to evaluate the relationship between levels of afterschool attendance and academic and behavioral outcomes. To complete this analysis, groups of regular program attendees (30+, 60+, and 90+) were matched with demographically similar comparison groups. Since academic data from the previous year was not available for all participants, the matched-groups analyses are considered “exploratory” as opposed to a true randomized control trial. Interviews with eight grantees were also conducted to form case studies on the benefits and challenges of implementing 21st CCLC programs.
Evaluation Type: Quasi-experimental;Non-experimental
Summary of Outcomes:
Indiana’s program evaluation shows the positive impact of 21st CCLC programs on student academic performance through a comparative analysis of students based on levels of program attendance. In 2018, more than a third of program participants attending 30-59 days passed the English/language arts (ELA) (47 percent), math (42 percent), and science (36 percent) segments of the Indiana Statewide Testing for Educational Progress (ISTEP). For high-participating students (participants who attended the program 90+ days), a higher percentage of students passed the ELA (52 percent), math (48 percent), and science (41 percent) tests compared to those attending less frequently. *The ISTEP English/language arts and math tests measures the achievement of students in grades 3 through 8, while the science test measures achievement of students in grades 4 and 6. 3rd through 5th grade program participants largely contributed to the differences in ELA and math, and 4th grade program participants contributed most to the difference in science.
Students attending 90+ days of programming were also more likely to either have earned a grade of B or better, or have improved their grade from fall to spring compared with other program attendees. In English/language arts, 74 percent of students attending 90+ days earned a B or better or improved their grade compared with 63 percent of students who attended 1-29 days of programming; while in math, 72 percent of students attending 90+ days earned a B or better or improved their grade compared with 60 percent of students who attended 1-29 days. Overall, 21st CCLC participants attending 90+ days earned significantly higher average math and English/language arts grades in spring 2018 compared to students who attended fewer days.
The evaluation also calculated the number of years that participants have been attending programing for 60 or more days. On average, participants who attended 60 or more days in the program for a period of 3 to 4 years had significantly higher ELA and math grades at the end of the 2018 school year and had significantly higher ELA, math, and science ISTEP passing rates compared to participants attending less frequently.
Teacher surveys also reflect a positive relationship between 21st CCLC participation and student academic performance and behavior. Teachers reported that more than 6 out of 10 students attending 60 or more days and identified as needing to improve in the following respects, improved their academic performance (68 percent), classroom participation (64 percent), and quality of homework completion (63 percent). More than half of students who were identified as needing improvement by teachers in the following regards also improved in being attentive in class (57 percent), coming to school motivated to learn (56 percent), their behavior in class (53 percent) and getting along with their peers (52 percent).
Additionally, the evaluation found a statistically significant relationship between program participation and school day attendance, where students who attended the program at least 90 days missed less days of school than their peers who attended the program less often.
Date Added: September 3, 2019
