Evaluation
0
Communities Organizing Resources to Advance Learning (CORAL) Initiative Print E-mail

 

Click to go directly to: Goal
Description
Participating grantees
Evaluation
Publications

 

Goal

To increase the academic achievement of youth, with an emphasis on elementary school students, by involving students, families, schools and organizations in high-quality out-of-school learning opportunities, in five California cities.

Description

The James Irvine Foundation launched the eight-year CORAL after-school initiative in 1999 with the goal of helping to improve the academic achievement of children in the lowest-performing schools in five California cities: Fresno, Long Beach, Pasadena, Sacramento and San Jose.

Once fully operational, this large-scale initiative served approximately 5,000 children each year — over half of whom were designated as English learners and many of whom came from low-income families — across more than 30 school- and community-based sites. Most of the youth were of elementary-school age, primarily first to fifth-graders, with a small proportion in middle-school grades. The Foundation provided implementation support in all of the cities, with the objective of funding the initiative for five to six years in each site. In total, the Foundation committed more than $58 million to CORAL, making it the most significant and ambitious initiative undertaken by Irvine.

Following disappointing outcomes identified through a midpoint review, CORAL focused the wide breadth of programs offered at its sites on literacy activities and boosted program quality through a rigorous process of continuous improvement and staff development. These changes led to pronounced gains in achievement for a range of students.

The children involved in CORAL represented great diversity in their ethnicity and language proficiency and also, to some degree, in their performance at school. This diversity adds dimension to an examination of the role that after-school programs can play in the lives of different subgroups of youth and, in particular, English learners — a topic often missing in after-school research.

CORAL offers several key lessons to those with a stake in the success of after-school programs. Chief among the lessons are that after-school programs can, indeed, help promote student academic achievement, and that success requires targeted investment, stakeholder commitments, focused academic support, quality programming, and a process of continual improvement to attain and maintain high levels of quality.

Participating grantees

The lead agencies in each CORAL city are:

Evaluation

During the school years from 2004 to 2006, the Foundation engaged Public/Private Ventures (P/PV) to evaluate the CORAL initiative. P/PV is a national nonprofit organization whose mission is to improve the effectiveness of social policies, programs and community initiatives, especially as they affect youth and young adults. P/PV served as a key independent partner in helping to reorient the focus of CORAL after a midcourse assessment revealed disappointing outcomes. P/PV also brought a rigor and discipline in implementing these changes, helping to pave the way for the initiatives' eventual successes, including achievement of measured gains in reading levels by participants receiving consistent, quality literacy programming after school. Along the way, P/PV documented CORAL outcomes, lessons learned and promising strategies for boosting student achievement through after-school programming.

Objectives:

The evaluation of CORAL was designed to accomplish three objectives:

  1. Document the extent to which CORAL was successful in reaching its ultimate goal of improving young people's academic achievement, particularly in the area of literacy development.
  2. Examine several key questions of significant interest to the broader after-school field. There is particular interest in the field regarding recruitment and retention of student participants, quality of the activities provided, and outcomes. Across the country, programs struggle with issues of participation and are unsure about what constitutes high-quality after-school activities — especially those that provide literacy or other academic instruction. The CORAL evaluation addressed these issues.
  3. Provide feedback to the CORAL cities regarding the sites' progress toward meeting benchmarks of quality implementation and outcomes along the way. Doing so helped the cities and sites take stock of their progress, including both their accomplishments and the challenges they face, so they could make midcourse corrections to strengthen their programs.

Timeframe:

September 2004–September 2006

Methods:

Multiple methods were used to collect information for the evaluation. Data was collected across all sites and youth involved in CORAL, as follows:

  • A "management information system", or MIS, collected enrollment and participation information on all youth enrolled in CORAL, which was critical for documenting CORAL's success in meeting its goals for recruitment, participation and retention, and in understanding whom CORAL serves. P/PV also gathered longitudinal school records for all youth to examine possible changes in achievement and school attendance linked to duration and intensity of participation in CORAL.
  • A P/PV evaluation team visited each site twice a year to conduct interviews with CORAL staff and collaborating partner staff to understand implementation approaches, successes and challenges, and sustainability efforts. In addition, P/PV conducted focus groups with CORAL parents during their site visit to learn about parents' goals for and experiences with the CORAL programs.
  • A staff survey was administered to CORAL and partner agency staff in fall 2004 and was repeated in spring 2006 to CORAL and partner agency staff to learn about staff training and experiences that might relate to program outcomes.
  • A cost survey was administered in spring 2005 and was repeated in 2006 to understand issues of funding and sustainability.

For a subset of youth (those enrolled in third and fourth grade in fall 2004), the evaluation collected more in-depth information:

  • Youths outcomes were measured through pre/post literacy assessments, longitudinal school records, pre/post surveys of youth and teachers (to assess changes in attitudes and behaviors toward school and toward reading, relationships with peers and adults, and experiences in CORAL), and a one-time parent survey in spring 2006.
  • Observations of third/fourth grade CORAL activities over fall/spring 2004/2005 and observations of fourth/fifth grade CORAL activities over fall/spring 2005/2006 were made to assess the quality of the activities, with a particular focus on literacy activities, in order to be able to link activity quality to youths participation, retention and outcomes.
  • The same group of third and fourth graders was followed for spring 2006 surveys and assessments, whether or not the students continued to participate in CORAL.

Findings:

The results of CORALs evaluation are informative for program designers, funders, researchers and policymakers interested in making after-school programs as effective as possible for children. Summarized here in brief, key initiative findings (and related implications) are described in greater depth within the CORAL series of publications.

Findings include:

  • Children's reading success was strongly related to literacy programming quality. For example, children exposed to quality program implementation gained 0.45 grade-levels; others gained 0.26 grade-levels. And, the quality of CORAL programs was increased relatively quickly.
  • Higher levels of engagement were attained and are related to positive changes in children's attitudes toward reading as well as attitudes and behaviors in school, such as children's enjoyment of reading, whether they liked and wanted to go to school, and the time they spent reading after school.
  • English learners and children far behind in reading each showed similar gains when compared to their CORAL peers. Fifty-three percent of participants were English learners.

Products:

Evaluator:

Amy Arbreton
Director of Research
Public/Private Ventures (P/PV)
Lake Merrit Plaza
1999 Harrison Street, Suite 1550
Oakland, CA 94612
www.ppv.org

top of page

0