The Afterschool Alliance has produced a variety of evaluation-related materials over the past 15 years. Some provide an overview of the impacts of afterschool, before-school and summer learning programs, while others dig into specific areas of interest, including science, technology, engineering and math (STEM) and social emotional learning.
Afterschool and summer learning programs inspire students to learn, keep children and teenagers safe, and give parents peace of mind. This fact sheet provides a small sampling of findings—from meta-analyses to program-level evaluations—that show that there is demonstrable evidence of the positive impact programs have on students’ academics, school-day attendance, engagement in learning, and behavior.
21st Century Community Learning Centers (21st CCLC) are afterschool and summer learning programs that are locally-designed school and community solutions that help kids learn and grow, keep children and teenagers safe, and support families to balance work with home. This fact sheet provides a small sampling of findings from evaluations of the federally-funded Community Learning Centers programs that demonstrate the positive impact programs across the country are having on students’ academics, school-day attendance, engagement in learning, and behavior.
Glossary
This glossary includes terms used in the Afterschool Impacts Database to help you navigate the search function and find the information that best suits your needs. Suggestions for additional terms to include in the glossary? Email Us!
21st Century Community Learning Centers (21st CCLC): The 21st CCLC initiative, established in 2002, is the only federal funding source dedicated exclusively to before-school, afterschool and summer learning programs. Each state education agency receives funds based on its share of Title I funding for low-income students at high-poverty, low performing schools. Funds are also allocated to territories and the Bureau of Indian Affairs. 21st CCLC programs provide academic enrichment and a broad array of additional services designed to complement the regular academic program, such as art, music, recreation programs, STEM (science, technology, engineering and math) programs, and character education programs. Evaluations were classified as 21st CCLC if the evaluated program(s) was a recipient of this funding stream.
Academic: Evaluations were classified as Academic if they included academic-related indicators, such as grades, test scores, or improved competency reflected by teacher or student surveys. On-time grade promotion and graduation from high school were also included as Academic. Academic areas of focus include, but are not limited to, literacy and science, technology, engineering, and math (STEM).
Attendance & Behavior: Evaluations identified as Attendance & Behavior include outcomes related to changes in school day attendance, school day behavior or behavior in the afterschool program. Behavioral outcomes include, but are not limited to, changes in homework completion, disciplinary incidents, peer relationships, or interest and engagement in school.
Children of Color: Children of Color includes African-American, Hispanic, Asian American/Pacific Islander, Native American or bi/multi-racial students. Evaluations were designated as Children of Color if more than 50 percent of the evaluated program(s) participants were children of color, or if the evaluation included outcomes specific to children of color.
College & Career: Evaluations designated College & Career include outcomes related to preparing students to be successful in college or in future careers. College and career outcomes include, but are not limited to, increased knowledge of college and career opportunities, increased understanding of skills needed for college and career success, or increased ability to complete career-necessary skills.
Educator Outcomes:Educator Outcomes include evaluations that report on outcomes related specifically to the impacts on the educators who provided programming.
English Language Learners (ELL):English language learners are students whose primary language is not English, are currently working to develop their English, and often benefit from additional supports to improve their English reading, speaking, writing and comprehension skills. ELL students currently make up 9.3 percent of K-12 students in the United States. Evaluations were classified as ELL if more than 9.3 percent of students in the program(s) were designated ELL or if the evaluation included outcomes specific to ELL students.
Experimental: In an experimental study, evaluators randomly assign individuals to a treatment group and control group before they began the evaluation. In the case of afterschool evaluations, the treatment group would be the group of students who are assigned to participate in the afterschool, before-school or summer learning program, while the control group would be the group of students who do not participate in the program. Experimental design studies are considered the gold standard of evaluation because their randomized design process is the best way to determine the effectiveness of a program or policy. Under the Every Student Succeeds Act (ESSA), experimental design evaluations that show positive and statistically significant results would fall within the "Strong" tier of evidence-based practice.
Girl-Focused: Evaluations were designated as Girl-Focused if the evaluated program(s) only served girls, self-identified as Girl-Focused, served a high percentage of girls, tailored their program to girls, purposefully recruited girls, or included outcomes that were specific to female participants.
Health & Wellness: Evaluations designated as Health & Wellness include outcomes related to improved overall student health, including: physical fitness, healthy eating habits, or reductions in obesity.
High-Poverty: Evaluations were designated High-Poverty if 75 percent or more of students in the evaluated program(s) were eligible for the Free and Reduced Price Lunch program, which is commonly used as a determinant for families living in or near poverty, or if the evaluation included outcomes specific to youth from low-income families.
Non-Experimental: Non-experimental refers to studies that collect information to observe and measure what is taking place. For example, surveys or polls to describe opinions, case studies, and longitudinal research observing the same group of individuals over an extended period of time are all non-experimental research designs. Under the Every Student Succeeds Act (ESSA), non-experimental design evaluations that are correlational—meaning they look at the relationship between two variables—and that also control for selection bias would fall within the "Promising" tier of evidence-based practice.
Quasi-Experimental: In a quasi-experimental study, evaluators compare participants in the program (also referred to as the treatment group) to a similar group of students who did not participate in the program (also referred to as the control group). However, a key difference between a quasi-experimental design study and an experimental design study is that the quasi-experimental design study does not randomly assign individuals to the treatment group or control group. For instance, evaluators may compare spring test scores of afterschool program participants to those of their peers who are not participating in the program. Frequently the participant and non-participant groups will be matched by community type, gender, and socio-economic background to help create comparison groups that are as similar as possible. Under the Every Student Succeeds Act (ESSA), quasi-experimental design evaluations that show positive and statistically significant results would fall within the "Moderate" tier of evidence-based practice.
Rural: Evaluations were designated Rural if the program(s) evaluated self-designated as serving a rural community or the evaluation contained outcomes that were specific to rural communities.
Scope: The scope of the evaluation refers to the area that the evaluation covers. For instance, some evaluations look at programs from multiple cities (multi-city), others review a program in one location (local), and others evaluate programs across a state (statewide) or the nation (national).
Social and emotional learning (SEL): Evaluations were classified as SEL if they include outcomes related to social and emotional learning. SEL focuses on the knowledge and skills necessary to manage emotions, set and achieve positive goals, feel and show empathy to others, build positive relationships and make responsible decisions. Terms also used to describe SEL skills include, but are not limited to, 21st century skills, soft skills, non-cognitive skills and employability skills.
Special Populations:Special Populations refer to student groups who are from traditionally underserved communities and/or are at risk of falling behind in school. Special Populations include: English language learners, children of color, low-income students, girls, or students with special needs and disabilities.
STEM:STEM refers to the subjects of science, technology, engineering and math.
Suburban: Evaluations were classified as Suburban if the program(s) evaluated self-designated as serving a suburban community or the evaluation contained outcomes that were specific to suburban communities.
Title I Eligible: Selecting Title I Eligible will include evaluations of programs where at least 40 percent of students participating in the program are eligible for the Free and Reduced Price Lunch program, which is commonly used as a determinant for families living in or near poverty.
Urban: Evaluations were classified as Urban if the program(s) evaluated self-designated as serving an urban community or the evaluation contained outcomes that were specific to urban communities.
Afterschool Alliance Evaluation Resources
BLOGS
Evaluating Afterschool, Part 1: Tips for Getting Started from Dallas Afterschool(September 2016)
In the first blog of the series, "Evaluating Afterschool," which asks afterschool program providers to share tips and lessons learned on their evaluation journey. This blog is by Rachel Johns, the research and evaluation manager at Dallas Afterschool.
Evaluating Afterschool, Part 2: Evaluation as a Mission-Driven Investment(November 2016)
In the second blog of the series, Jason Spector, senior research & evaluation manager for After-School All-Stars, explain why evaluation is beneficial to afterschool programs. He also shares his tips for how to decide if your organization is ready to evaluate!
WEBINARS
Measuring the Impact of STEM Learning in Afterschool(November 2016)
How can we understand and measure the true impact of afterschool STEM experiences? In this webinar we will hear about three innovative efforts—Connected Learning, Activated Learner, and Longitudinal—that are studying how learning, particularly STEM learning, develops across time and space, and how afterschool programs contribute to that process.
Are You Ready to Assess Social and Emotional Development?(May 2016)
In this webinar, Deb Moroney and Michael McGarrah from American Institutes for Research (AIR) joined us to talk about Ready to Assess, a suite of tools that can help afterschool and education leaders, practitioners, and policymakers decide whether and how to assess social and emotional development. Kristin Romens, Senior Consultant at The Learning Agenda, discussed the progress and challenges educators face around implementing social and emotional learning best practices.
Creating the Conditions for Social and Emotional Learning(February 2016)
American Institutes for Research (AIR) spoke about their resources that support the application of best practices in social and emotional learning in afterschool. Also covered in the webinar is how the Expanded Learning 360/365 group, in collaboration with the California School-Age Consortium, is using AIR's Self-Reflection Tool for building quality in afterschool systems throughout California.
REPORTS
Taking a Deeper Dive into Afterschool-Positive Outcomes and Promising Practices(February 2014)
To better understand promising practices in the afterschool field, this report reviews outcomes associated with participation in afterschool programs, synthesizing high-quality evaluations of ten afterschool programs, explores research spanning hundreds of programs to present a summary of promising practices of afterschool programs, and provides specific examples of ways in which the afterschool programs employ each promising practice.
Examining the Impact of Afterschool STEM Programs(July 2014)
This paper summarizes evaluation data from a selection of strong afterschool STEM programs, providing a snapshot of the types of substantive impacts afterschool programs are having on youth. Check out our 2016 update for more programs and updated evaluation data.
A Framework of Youth Outcomes for STEM Learning(January 2013)
What kinds of student outcomes are reasonable to expect from afterschool STEM programs? This report presents a consensus set of developmental outcomes gleaned from the research literature, and afterschool experts and stakeholders. The framework also includes indicators of progress toward these outcomes, as well as types of evidence that can demonstrate the impact of STEM programming in afterschool.