Evaluator: Pennsylvania Department of Education and the Allegheny Intermediate Unit
Evaluation Methods: This evaluation used a Grantee Report created and implemented by the Allegheny Intermediate Unit along with federal 21st Annual Performance Report data (21APR) to examine three performance measures: “participants in 21st century programs will demonstrate educational and social benefits and exhibit positive behavior changes,” “increasing percentages of students regularly participating in the program will meet or exceed state and local academic achievement standards in reading and math,” and “students participating in the program will show improvement in the performance measures of school attendance, classroom performance, and reduced disciplinary referrals.” The evaluation also examined parent and student satisfaction from aggregate data collected and submitted by grantees.
Due to the transition from the Profile and Performance Information Collection System (PPICS) to the 21APR system, the report cautions to “exercise extreme caution in comparing the 2014-15 state evaluation to prior or future years.”
Evaluation Type: Non-experimental
Summary of Outcomes: Student outcomes were assessed using the 21APR system reports and the PA Grantee reports. Overall, the evaluation found students regularly participating in afterschool programs saw gains in reading and math, as well as improvements in school behavior and school day attendance. Surveys of students attending the programs and their parents found that both were very satisfied with programs overall.
Report card analysis from the 21APR system showed that 44 percent of all regular attendees (students that attended the program more than 30 days) improved in reading, and 43 percent of all regular attendees improved in math. Of the 53 grantees that reported local reading and math results through the PA Grantee Report, the majority of grantees reported that more than 25 percent of students showed improvement. More specifically, 37 percent of students showed improvement in reading and 46 percent of students showed improvement in math.
The evaluation also found that students attending the program 60 to 89 days had the highest percentage of improvement in their reading (48 percent) and math (45 percent) report card grades from fall to spring and that students who regularly attended the program performed better on local reading and math assessments than their peers who did not attend the program regularly (less than 30 days).
Seventy-three grantees chose to submit data on behavior. Among regularly attending students that needed to improve in the area of school behavior, it was reported that 38 percent of students improved. Ninety-three grantees reported attendance outcomes. Among students who regularly attended the program who were in need of improving their school day attendance, 42 percent saw an improvement in their attendance. Teacher reported behavior improvement outcomes from the 21APR system showed that 47 percent of all regular attendees’ behavior improved, and 56 percent of regular attendees showed improved homework outcomes. Ninety-three grantees reported whether students had successfully participated in school credit recovery programs, and of the 2,702 students that had attempted credit recovery, 74 percent recovered one or more courses or credits.
Student and parent satisfaction was assessed from aggregate responses to survey questions reported by grantees. Overall, 59 grantees reported on student feedback. Among students surveyed, almost all students reported that they were satisfied with their program overall (95 percent), with a strong majority of students reporting that they were very satisfied (67 percent). Parents were also satisfied with their students’ experience, with 81 grantees reporting parent feedback results. Ninety-eight percent of parents reported that there were satisfied with the program overall, with 83 percent of parents saying they were very satisfied. Furthermore, parents reported that they were very satisfied with their child’s results related to academics (76 percent) and recreation within the program (83 percent). Almost all parents surveyed reported that they agreed that the program offered their child a variety of academic and enrichment activities (98 percent).