Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
Maine Department of Education 21st Century Community Learning Centers Program, Statewide Evaluation Report: 2015-2016
Year Published: 2017
A statewide evaluation of Maine’s 21st Century Community Learning Center (21st CCLC) programs during the 2015-2016 school year examined academic and behavioral outcomes associated with regular participation in the program among students who were designated as low-performing in school. The evaluation found that a majority of regularly attending low-performing students increased both their math and English language arts/literacy test scores. Teacher surveys showed that the majority of regularly participating low-performing students improved in-class behavior from the fall and spring semesters. Student surveys showed that the majority of students learned something new at their 21st CCLC program and enjoyed attending their program.
Program Name: Maine’s 21st Century Community Learning Centers
Program Description: Maine’s 21st Century Community Learning Center program—which has received federal support through the 21st CCLC initiative starting in 2003—serves high-needs communities across the state, providing local afterschool and summer programming to 23 grantees operating 64 centers during the 2015-16 school year. Across all sites, 8,454 students were served, approximately 75 percent of whom were designated as a “low-performing student.”
Scope of the Evaluation: Statewide
Program Type: Afterschool
Location: Maine
Community Type: Rural, Urban, Suburban
Grade level: Elementary School, Middle School, High School
Program Demographics: All schools served by Maine’s 21st CCLC programs are Title I eligible, meaning more than 40 percent of students qualify for Free and Reduced Price Lunch. Of students participating in Maine’s 21st CCLC program, 68 percent qualified for Free and Reduced Price Lunch and 75 percent were designated as “low-performing,” where “low-performing” was defined as “students that did not meet the academic standards required by the for their grade level upon enrollment in the program, as evidenced by standardized assessment results.” 15 percent of participants identified their race or ethnicity as non-white.
Program Website: http://www.maine.gov/doe/21st-century/
Evaluator: Madden, P., Hart, P., & Charles, J. Market Decisions Research.
Evaluation Methods: This evaluation used student math and English language arts (ELA)/literacy scores, student surveys, teacher surveys, and program reports to assess the academic and behavioral impact of regular participation in 21st Century Community Learning Center programs in Maine. Chi-squared tests of independence were used to assess academic outcomes in relation to student and teacher survey responses on program engagement and in-class behavior. Logistic regression was used to determine the relationship between program characteristics and math and ELA/literacy test score improvement.
Evaluation Type: Non-experimental
Summary of Outcomes:
The evaluation of Maine’s 21st CCLC programs during the 2015-16 academic year found that low-performing students who regularly participated in the programs—students who participated in the program at least 30 days—showed improvements in academics, homework completion, and classroom participation and behavior. Regarding behavior and engagement in learning, the majority of regularly attending low-performing students made improvements in positive classroom behaviors and engagement in learning. For instance, based on teacher surveys, the percentage of students who took on tasks that contributed to the classroom increased from 58 percent in the fall of 2015 to 65 percent in the spring of 2016, the percentage of students that were able to regain control of their behavior when warned increased from 74 percent to 77 percent, and the percentage of students that had constructive contributions to class discussion increased from 57 percent to 63 percent.
Chi-squared tests of independence on teacher survey results and math scores found that positive teacher survey responses on student behavior were significantly associated with improved math assessment scores (p<0.05). Regarding improved ELA/literacy scores, the only student behavior that was positively associated was usually or always contributing constructively to class discussions.
Examining regularly attending low-performing students’ math and English language arts (ELA)/literacy assessment scores, 79 percent increased their math scores and 78 percent increased their ELA/literacy assessment scores. Logistic regression estimates showed that participating in mentoring activities was associated with increases in students’ math and ELA/literacy scores.
According to student surveys, approximately 9 out of 10 regularly attending low-performing students reported that they enjoyed the program (91 percent) and had fun when at the 21st CCLC program (91 percent). Students also reported that they learned new things in their afterschool program (79 percent), they were able to do things at their afterschool programs that they had never done before (76 percent), and that they felt challenged in a good way when at the program (73 percent).
Associated Evaluation: http://www.maine.gov/doe/21st-century/documents/Maine21stCCLCEvaluationReport-01.09.2017.pdf
