To strengthen the ability of California museums to contribute to the education of young people during out-of-school hours and, ultimately, to improve their academic achievement.
Between 2000 and 2004, Irvine provided nearly $4 million to 10 museums to develop and evaluate after-school programming for young people through the Museum Youth Initiative (MYI). The initiative's hypothesis was that if museums delivered high-quality educational programs for children and youth, intermediate outcomes would include improved learning behaviors, attitudes, and skills. Long-term outcomes would include improved student academic achievement in specific curriculum areas. The initiative also sought to help participating museums institutionalize youth development principles, become learning environments that provide academic enrichment, and sustain program practices and resources over the long term.
Irvine contracted with Museum Management Consultants to conduct an evaluation of the Museum Youth Initiative to assess the progress of participating museums, help them refine their programs, and determine if the initiative's goals to improve the academic achievement of initiative participants were achieved.
The executive and program staff of participating museums were the principal audiences for this evaluation. Throughout the initiative, the evaluation provided the museums with helpful benchmark data across the initiative cohort and insights into effective program strategies. The reports served to facilitate reflection by museum staff about their progress and stimulate program improvements at several museums. The evaluation was less successful in assessing program impact and making a direct connection between student outcomes and program elements. In large part, this was due to resource constraints and related design limitations, as well as significant variation across museums in terms of program length, offerings, and youth population served.
The evaluation used both quantitative and qualitative methods. The evaluation team made multiple site visits to observe program activity, conducted interviews, and surveyed youth participants, their teachers, parents, and museum program staff. In addition, Museum Management Consultants gathered and analyzed participation data from each of the participating museums.
The initiative programs successfully targeted those youth who are often more difficult to reach and serve. Two-thirds of participating students were students of color. Nearly one-third were elementary students; two-thirds were in middle school or high school. Two-thirds of participating students were considered “educationally disadvantaged” and in need of extra academic support.
Consistent with other evaluation studies in the after-school education field, the evaluation documented that students who had higher attendance rates and more exposure to the museum programs experienced better outcomes in terms of improved study skills and improved higher-order thinking. It also showed that disadvantaged youth made greater gains in those areas than other youth in the same education and enrichment programs.
The museum programs were able to engage students in learning. The evaluation indicated that museums can play a value-added role as effective informal learning organizations. The evaluation further suggested that the museum after-school programs could help improve student thinking skills, behaviors, and attitudes. On the other hand, the evaluation also indicated that the after-school programs were unlikely to improve formal, in-class achievement, as reflected in grades and teacher ratings. Improvements in grade point averages among participating students were of a relatively low magnitude in any one year and inconsistent across all years.
From a more qualitative perspective, the evaluation documented experiences and lessons of interest to the broader museum field. Based on interviews with museum leaders and practitioners, there was particular interest in the field regarding recruitment and retention of youth participants, program quality, outcomes, and financially sustaining museum youth programs. The final report on the Museum Youth Initiative, Museums After School, expanded upon the evaluation information to address these issues and also the larger questions of the unique assets that museums bring to after-school programming and in offering high quality learning opportunities for young people.