Afterschool Impacts Database
Evaluation 101
Learn more about the basics of evaluation, as well as how to find an evaluator for your program.
Afterschool Evaluation
View Afterschool Alliance resources, including a glossary of terms used in the database, evaluation-related blogs, webinars and more.
Additional Resources
Our list of evaluation resources from other organizations, including how to collect and work with data.
Afterschool Research
Want to find what we know about afterschool programs more broadly, not just individual programs? Head to our Afterschool Research page!
The Impact of Afterschool STEM: SHINE After School Program
Year Published: 2016
SHINE (Schools & Homes in Education) is a comprehensive afterschool program that provides academic and social support to youth in a primarily rural region of Pennsylvania. SHINE offers STEM throughout the grades, with the intention of building a STEM pipeline from kindergarten to career. In this selection of evaluation data from the 2013-2014 school year, participants demonstrated gains along three major categories of youth outcomes—interest in STEM, capacity to engage in STEM, and finding value in STEM.
Program Name: SHINE After School Program
Program Description: SHINE (Schools & Homes in Education) is a comprehensive afterschool program that provides academic and social support to youth in a primarily rural region of Pennsylvania. SHINE offers STEM throughout the grades, with the intention of building a STEM pipeline from kindergarten to career.
Scope of the Evaluation: Local
Program Type: Afterschool
Location: Carbon & Schuylkill Counties, PA
Community Type: Rural
Grade level: Elementary School
Program Demographics: 87 percent of elementary students qualify for free or reduced price lunch; 23 percent are English Language Learners; 31 percent have special needs or disabilities; 10 percent African American, 23 percent Hispanic and 67 percent Caucasian.
Program Website: http://shineafterschool.com/shinewordpress/
Evaluator: SHINE staff
Evaluation Methods: Outcomes are measured using several data sources, including surveys of staff, school day teachers, and parents; pre- and post- surveys of participants; the Common Instrument from the Program in Education, Afterschool and Resiliency (PEAR) at Harvard University; report cards; and state assessment results. To assess program quality, site observations are conducted.
Evaluation Type: Non-experimental
Outcomes:
Below is a selection of evaluation data reported by the program around three major categories of youth outcomes—interest in STEM, capacity to productively engage in STEM, and finding value in STEM. These outcomes are an excerpt from a 2016 Afterschool Alliance paper, "The Impact of Afterschool STEM: Examples from the Field."
- Almost three-quarters of SHINE students like science (73 percent) of SHINE and get excited to find out that they will be doing a science activity (76 percent).
- Participants exhibit a strong curiosity about science—88 percent agree that they like to see how things are made, 72 percent want to learn more about STEM, and 74 percent want to understand science (for example, to know how computers work, how rain forms or how airplanes fly).
Capacity: I can do this
- All elementary students in SHINE are referred to the program based on academic need, and as a result of participation, students are highly successful in their school science classes—94 percent received passing grades, and 57 percent received an A or B. This is supported by parents, as 82 percent indicated their child improved in science after participation.
- Additionally, 82 percent of students received passing grades math, with 46 percent receiving an A or B.
Value: This is important to me
- Youth could picture a future in STEM—91 percent said that science and math will be used in their future career and 52 percent would like to have a science or computer job in the future.
- Participants gained an understanding of the specifics of STEM careers—80 percent reported that they know what an engineer does and 94 percent understood that engineers need to know both math and science.
Associated Evaluation: http://afterschoolalliance.org/documents/AfterschoolSTEMImpacts2016.pdf
