Credit: Allison Shelley for American Education
California’s $4.1 billion investment in community schools is the largest in the nation. An investment of this size raises important questions about whether community schools are working and what difference they are making for students.
Community schools are intended to provide the multitude of opportunities and supports that students need to thrive and succeed. They include a rich array of integrated services, expanded learning opportunities, deep community partnerships, and importantly, offer a more democratic way of engaging with students, families and the school community to shape school priorities and vision.
Community schools are a complex endeavor that, when done well, substantially expand what schools do to support students — and who is included in this work. Assessing the implementation and impact of community schools is similarly complex.
The California Department of Education (CDE) recently requested proposals for an evaluator of the California Community Schools Partnership Program (CCSPP) initiative and will make a selection this spring. We wish to share lessons for future evaluators of program, the department of education, and the county offices of education, districts, schools and communities implementing these community school models throughout the state and country. Ideally, whatever data is required for the state evaluation and grant compliance should also be usable to help schools and districts in guiding strategic, high-quality community schools implementation.
The suggestions below come out of our work as evaluators for Oakland Unified School District’s community schools initiative for many years while working at Stanford University’s Gardner Center, and as authors of a book about the effort to transform 86 of the district’s schools into community schools.
- Community schools are not a program that a school either has or does not have, but rather an approach to education with many gradations along a spectrum.
While many California schools have recently or will soon receive funding to become community schools, fully implementing the model can take years. Further, many of these schools already operated some elements of community schools prior to funding (such as expanded learning, school-based health services, positive discipline practices, coordination of services, or family engagement strategies), without the “community school” label. The community school grant, which includes funding for an on-site community school coordinator, is meant to expand and strengthen whole-child work and bring increased collaboration and coherence across many people, organizations and initiatives. Thus, identifying the community school “start date” as the receipt of CCSPP funds is not as clean as it may seem.
Lesson for evaluators: The multifaceted and fluid nature of community schools make traditional causal research designs challenging. Evaluators ideally should adopt a mixed-methods (qualitative and quantitative) approach that examines change over time at community schools and illuminates connections between quality implementation and desired outcomes. Evaluators should thoughtfully consider the extent to which it is possible to isolate the impact of community schools and be precise about which elements or stages of community schools are captured in any assessment of impact.
- A multilevel strategy map can provide a framework to guide implementation and evaluation.
Community schools provide a range of additional services, engage families and community organizations, and align all of these toward school goals; increasing students’ well-being and, ultimately, educational success. Successful community schools are more than a site-level intervention and require intentional district support. Given the multifaceted nature of community schools, we recommend a theory of change or “system strategy map” at three levels. Assessing key activities and outcomes at the 1) System (school district), 2) Site (school and community), and 3) Individual (student and family) levels can help ensure a comprehensive evaluation and improve understanding of differences in implementation and outcomes across the state.
Lessons for evaluators: Consider grounding your evaluation in a theory of change, and incorporating strategies and outcomes at individual, setting, and system levels.
- Impact on traditional measures of student success can take time, and is predicated on quality implementation. But there is a lot you can measure along the way.
Community schools are a whole child, whole school improvement strategy. It takes time to adopt new practices, integrate resources, cultivate meaningful collaboration, develop supportive structures, and shift culture. We are unlikely to see immediate effects on traditional measures of student achievement — e.g., test scores, graduation rates, attendance, and suspensions — for at least 3-5 years. We may start to see bumps in achievement for specific student subgroups as community schools are designed to precipitate more equitable access across opportunity gaps.
To impact long-term student wellbeing and success, quality implementation matters. Proximal indicators can show if schools are on the right track: for example, participation, knowledge, and use indicators (e.g., to what extent are students and families accessing services and opportunities; to what extent is staff aware of and utilizing community school resources); culture/climate indicators (e.g., levels of trust, collaboration, and participation); and if other enabling conditions are being met. Additionally, qualitative data is crucial for answering critical questions about how community schools are working, what is going well, what is not, and why.
These findings can directly inform program improvement at the LEA and state level. For example, some of our early research with Oakland Unified showed that many principals were struggling to understand their role in community schools development. In response to these findings, the district increased investment in professional development for site leaders.
Lessons for evaluators: Before assessing whether community schools are yielding desired results for students, it’s imperative to examine the extent to which implementation is happening as hoped and planned, such as, school-level coherence and collaboration and family-school partnerships. Further, an evaluation should include more nuanced indicators of student experiences beyond what is included in the California Data Dashboard and existing statewide culture/climate surveys to capture youth voice, cultural relevance and community connection.
The California Community Schools Partnership Program evaluator will set the tone for “what matters” in community school implementation across the state. Additionally, the evaluation activities should include support for schools, districts and county offices to help them use data in collaborative, participatory ways with their teams and community.
A strong evaluation of the California community schools initiative will provide lessons that inform ongoing school and district-level implementation, and give us an understanding of the difference community schools make for students and families.
•••
Kendra Fehrer is founder and principal of Heartwise Learning, which helps schools and organizations create practical, research-informed solutions to improve student learning and well-being.
Jake Leos-Urbel is senior director of learning and evaluation at Oakland Thrives. They are authors of the book “The Way We Do School: The Making of Oakland’s Full-Service Community School District”
The opinions in this commentary are those of the authors. If you would like to submit a commentary, please review our guidelines and contact us.
