Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
21st Century Community Learning Centers 2014-15 State Evaluation Report
Year Published: 2016
An evaluation of 21st Century Community Learning Centers (21st CCLC) programs in Pennsylvania using federally reported 21APR data (where APR stands for “Annual Performance Report” and the 21APR system collects information on 21st CCLC grantees and centers) and PA Grantee Reports to examine attendance and behavior; academic achievement in reading, math, and credit completion; and student and parent program satisfaction. The evaluation found that among students who regularly attended the program, 44 percent improved their reading grade from fall to spring and 43 percent improved their math grade from fall to spring. Based on teacher reports, 47 percent of regular program attendees improved their in-school behavior, and 56 percent of attendees improved homework outcomes. Overall, parents and students were both overwhelming “very satisfied” with their, or their child’s, experience in the program.
Program Name: Pennsylvania’s 21st Century Community Learning Centers
Program Description: Pennsylvania’s 21st CCLC program—which has received federal support through the 21st CCLC initiative—serves high-needs communities, providing academic enrichment through local afterschool and summer programming to 163 grantees during the 2014-15 program year.
Scope of the Evaluation: Statewide
Program Type: Afterschool
Location: Pennsylvania
Community Type: Rural, Urban, Suburban
Grade level: Elementary School, Middle School, High School
Program Demographics: Grantees of 21st CCLC funding in Pennsylvania are programs that target at-risk students who are performing below proficient academically.
Program Website: http://www.education.pa.gov/K-12/21st%20Century%20Community%20Learning%20Centers/Pages/default.aspx#tab-1
Evaluator: Pennsylvania Department of Education and the Allegheny Intermediate Unit
Evaluation Methods: This evaluation used a Grantee Report created and implemented by the Allegheny Intermediate Unit along with federal 21st Annual Performance Report data (21APR) to examine three performance measures: “participants in 21st century programs will demonstrate educational and social benefits and exhibit positive behavior changes,” “increasing percentages of students regularly participating in the program will meet or exceed state and local academic achievement standards in reading and math,” and “students participating in the program will show improvement in the performance measures of school attendance, classroom performance, and reduced disciplinary referrals.” The evaluation also examined parent and student satisfaction from aggregate data collected and submitted by grantees.
Due to the transition from the Profile and Performance Information Collection System (PPICS) to the 21APR system, the report cautions to “exercise extreme caution in comparing the 2014-15 state evaluation to prior or future years.”
Evaluation Type: Non-experimental
Summary of Outcomes:
Student outcomes were assessed using the 21APR system reports and the PA Grantee reports. Overall, the evaluation found students regularly participating in afterschool programs saw gains in reading and math, as well as improvements in school behavior and school day attendance. Surveys of students attending the programs and their parents found that both were very satisfied with programs overall.
Report card analysis from the 21APR system showed that 44 percent of all regular attendees (students that attended the program more than 30 days) improved in reading, and 43 percent of all regular attendees improved in math. Of the 53 grantees that reported local reading and math results through the PA Grantee Report, the majority of grantees reported that more than 25 percent of students showed improvement. More specifically, 37 percent of students showed improvement in reading and 46 percent of students showed improvement in math.
The evaluation also found that students attending the program 60 to 89 days had the highest percentage of improvement in their reading (48 percent) and math (45 percent) report card grades from fall to spring and that students who regularly attended the program performed better on local reading and math assessments than their peers who did not attend the program regularly (less than 30 days).
Seventy-three grantees chose to submit data on behavior. Among regularly attending students that needed to improve in the area of school behavior, it was reported that 38 percent of students improved. Ninety-three grantees reported attendance outcomes. Among students who regularly attended the program who were in need of improving their school day attendance, 42 percent saw an improvement in their attendance. Teacher reported behavior improvement outcomes from the 21APR system showed that 47 percent of all regular attendees’ behavior improved, and 56 percent of regular attendees showed improved homework outcomes. Ninety-three grantees reported whether students had successfully participated in school credit recovery programs, and of the 2,702 students that had attempted credit recovery, 74 percent recovered one or more courses or credits.
Student and parent satisfaction was assessed from aggregate responses to survey questions reported by grantees. Overall, 59 grantees reported on student feedback. Among students surveyed, almost all students reported that they were satisfied with their program overall (95 percent), with a strong majority of students reporting that they were very satisfied (67 percent). Parents were also satisfied with their students’ experience, with 81 grantees reporting parent feedback results. Ninety-eight percent of parents reported that there were satisfied with the program overall, with 83 percent of parents saying they were very satisfied. Furthermore, parents reported that they were very satisfied with their child’s results related to academics (76 percent) and recreation within the program (83 percent). Almost all parents surveyed reported that they agreed that the program offered their child a variety of academic and enrichment activities (98 percent).
