A project of the Afterschool Alliance.

Evaluation of Young Audiences of Maryland, 2016 Summer Arts and Learning Academies Program Goals

Year Published: 2016

This evaluation examines if during the summer of 2016, the Young Audiences Summer Arts and Learning Academy’s (YA Academy) program goals were met in the following areas: program attendance, improvement in math and writing skills, growth in social-emotional development, student enjoyment of the program, and parent satisfaction. Students who had high levels of participation in YA Academy saw gains in their math and writing skills and knowledge, as well as improvements in their social-emotional growth. Overall, this evaluation found that YA Academy was able to meet the majority of the goals they had set for summer 2016, meeting or exceeding their goals set for academic achievement in math and writing, students’ social-emotional growth, and parent satisfaction with the program.

Program Name: The Young Audiences Summer Arts and Learning Academy

Program Description: Young Audiences began in Baltimore in 1950, and in 2017 is the nation’s largest arts-in-education provider. As a Young Audiences affiliated program, the Young Audiences Summer Arts and Learning Academy (YA Academy) was created with the intention of reducing summer learning loss through arts-related learning, teaching literacy and math through the arts. During the summer of 2016, the free program was held at four sites across Baltimore City with locations at Gardenville Elementary, Thomas Jefferson Elementary/Middle, William Pinderhughes Elementary, and Fort Worthington Elementary/Middle schools.

Scope of the Evaluation: Local

Program Type: Summer

Location: Baltimore, MD

Community Type: Urban

Grade level: Elementary School

Program Demographics: Eighty-eight percent of students who participated in the Young Audiences Summer Arts and Learning Academy during the summer of 2016 were from high-poverty, Title 1 schools.

Program Website: http://www.yamd.org/

Evaluator: Stein, M. L.

Evaluation Methods: The evaluation collected data from program participants and included student program attendance, pre- and post-test adapted EngageNY math assessments and writing prompts, and pre- and post-test Devereux Student Strengths Assessment (DESSA), which assesses students’ social-emotional competencies. To measure student and parent perceptions of the program, students were given pre- and post-test surveys and parents given an exit survey. All of the pre-tests were administered during the first week of the program and the post-tests during the final week of the program. DESSA results are based on findings from ten randomly selected students from each classroom who administered the assessment.

Evaluation Type: Non-experimental

Summary of Outcomes: This evaluation found that students who had high levels of participation in the Young Audiences Summer Arts and Learning Academy (YA Academy)—attending at least 75 percent of the program—saw gains in their math and writing skills and knowledge, as well as improvements in their social-emotional growth. YA Academy was able to meet the majority of the goals they had set for summer 2016, meeting or exceeding their goals set for academic achievement in math and writing, students’ social-emotional growth, and parent satisfaction with the program. The program was less successful in meeting program attendance goals, and their goal for percentage of students who would recommend the program to their friends.
 
Examining academic achievement, the evaluation assessed the program’s impact on students’ math knowledge and skills and writing skills. Regarding students’ math achievement, the evaluation found that 79 percent of students—those who attended at least 75 percent of the program and took a pre- and post-test math assessment—had a positive change in their score from the first week of the program to the last week of the program. This exceeded the program’s goal of “75 percent of students who attended at least 75 percent of the program will show growth on the assessment from the first to the last week of the program.” Again looking at the outcomes for students who attended at least 75 percent of the program, the evaluation also found that 60 percent of the students who were scored on the pre- and post-test writing prompts showed improvements in the structure and content of their writing.
 
Of students who attended at least 75 percent of the program and who were administered pre- and post-test DESSA (96 students), 71 percent showed growth in either “goal-directed behavior,” “relationship skills,” or “self-awareness.” Approximately 30 percent of evaluated students showed growth in two or three of these areas.
 
Out of students who took the post-program survey (58 percent) that asked students about their program experience, 58 percent stated that they would recommend the program to a friend, 11 percent stated they would not recommend the program to a friend, and 33 percent chose not to respond. These results did not meet the initial goal of 80 percent of students recommending the program to a friend.  Parent satisfaction exceeded initial goals with 90 percent of parents rating their satisfaction between eight and ten on a scale of one to ten, and 95 percent of parents saying they would recommend the program to another parent.
 
The evaluation also found that 50 percent of students attended more than 74 percent of the program, which did not meet the goal of 75 percent of students attending 75 percent of the program.