Rethinking Measures of Success

Owner: Director of Program Impact & Visibility (DPIV) – with input from all Directors Audience: All OWL staff and stakeholders

This document outlines Open Way Learning’s detailed approach to measuring the impact of our work with schools and districts. Our mission is to catalyze learner-centered transformation by helping schools develop the conditions and systems necessary for sustained, equitable, and high-quality learning for every student.

Rather than narrowly focusing on test prep or high-stakes assessments, OWL uses a holistic, research-informed system of measurement aligned with deeper learning goals. We help schools integrate traditional academic metrics with meaningful indicators of student growth, engagement, and real-world readiness—measured over time through collaborative, iterative processes.

Guiding Principles

OWL's measurement philosophy draws from leading research and field-tested practices. Our approach emphasizes:

• Learning is multidimensional: Cognitive, emotional, and behavioral growth must all be considered. (Cantor et al., CASEL)

• Process data matters: Implementation quality and lived experience shape long-term results. (Carnegie Foundation)

• Improvement requires iteration: We use Plan-Do-Study-Act (PDSA) cycles to test and refine strategies. (Bryk et al.)

• Engagement drives outcomes: Environments that promote agency and connection lead to deeper learning. (Lucas Education Research, Hattie)

• Authentic learning yields results: Deeper learning and project-based strategies boost achievement, SEL, and postsecondary success. (AIR, Lucas Foundation)

Our goal is not to isolate a single cause of improvement, but to ensure the implementation of strategies known to work—customized to each context and evaluated using meaningful data.

Framing the Approach

Our method is anchored by one foundational question and three continuous improvement questions:

• Foundational: Have we clearly defined what success looks like—both short and long term?

• Improvement Questions:

  • Are we doing what we said we would do? (Process)

  • Are we doing it well? (Driver-level quality)

  • Is it producing the outcomes we want? (Impact)

This structure ensures that learning, not just accountability, is at the heart of the process.

We support schools in designing and refining learner-centered systems through co-design, local adaptation, and sustained cycles of reflection, iteration, and improvement. This includes applying tools from Design Thinking and Improvement Science—such as driver diagrams, PDSA cycles, and fidelity check-ins—to embed effective strategies in school culture.

Importance of Process Data vs. Outcome Data:

While outcome data (e.g. test scores, graduation rates) often dominate traditional data conversations, OWL’s work emphasizes the equal importance of process data— information that captures how well strategies are being implemented, how learners and educators experience new approaches, and how fidelity to intended practices evolves over time. Strong process data ensures we are not just measuring "whether" we are trending toward the desired outcomes we want, but also "how" success (or lack of success) was shaped by the design and delivery of our work. Prioritizing both process and outcome data fosters deeper learning, a continuous improvement mindset, and sustainable change at the school cultural level. OWL facilitates this approach through structured use of the Design Process and Improvement Science via PDSA cycles and driver diagrams to guide local, adaptive learning.

Key Concepts & Steps

1. Clarify Success Metrics at Multiple Levels

   • Begin by collecting empathy-driven data from students, educators, families, and community members to define what success looks like in their eyes.

   • Co-create a vision of learner success that reflects academic achievement, SEL growth, and career readiness.

   • Develop aligned process measures (e.g. implementation fidelity, student engagement, etc.) and driver measures (e.g. leadership practices, collaboration structures, etc.) that support the ultimate goals.

2. Co-Design and Localize Tools

   • Use OWL’s design sprints and planning protocols to co-develop contextually relevant tools—rubrics, portfolios, observational protocols, and community feedback loops.

   • Model and scaffold how to use these tools in ways that elevate student voice, build trust, and make learning visible.

   • Integrate these tools into daily practice through advisory programs, interdisciplinary units, and authentic assessments.

3. Prototype and Test

   • Apply Improvement Science strategies (PDSA cycles, driver diagrams, etc.) to launch small-scale innovations.

   • Establish norms for collecting qualitative and quantitative data during the test phase.

   • Ensure conditions for psychological safety and trust to support innovation without fear of failure.

4. Analyze and Share Findings

   • Reflect as a team using protocols and collaborative analysis tools to interpret data across artifacts, student reflections, teacher journaling, and walkthrough evidence.

   • Use OWL’s structured review frameworks to distinguish between bright spots, learning moments, and pivots.

   • Crosswalk findings with state accountability indicators and community priorities to ensure shared ownership and alignment.

5. Sustain and Scale

   • Shift from external facilitation to internal leadership through ongoing coaching, distributed leadership, and peer-driven PLCs.

   • Embed improvement science into school and district strategy cycles.

   • Support cross-school learning and build local capacity to continue iterating, even after direct support ends.

   • Align strategic growth with OWL’s philosophy of "scaling culture, not programs"—deepening impact without compromising authenticity.

Why This Matters

OWL’s approach helps schools move beyond short-term programs to foster a culture of innovation, equity, and reflective practice. We focus not on attribution, but on building the capacity for continuous improvement in complex, real-world environments. Our partners don’t just adopt strategies—they learn how to adapt, sustain, and scale them.

Rather than claim causation, we measure whether conditions are in place to enable known-effective strategies to work. We prioritize actionable data, contextual relevance, and student-centered measures that reflect what truly matters for long-term success.

Attachment 1: Examples of OWL-Supported Impact

OWL partners with schools and districts to catalyze learner-centered transformation by embedding high-quality Experiential Learning approaches, design thinking, and collaborative leadership into the fabric of school cultures. These partnerships are not one-time interventions but sustained co-design processes that foster authentic innovation rooted in empathy, equity, and educator empowerment. The impact on students across OWL-supported sites consistently reflects OWL’s mission to empower every learner through experiential, relevant, and equitable learning environments.

• Empowered Learning Through Student Voice and Authentic Engagement - Across multiple sites, students have demonstrated significant increases in self-direction, creativity, and academic agency. In Albuquerque Public Schools, students in the Summer STEAM program took ownership of their learning and applied real-world problem-solving skills in just 12 days of PBL-focused experiences. Teachers noted that students who were once disengaged or behaviorally challenged "blossomed" through meaningful, project-based work that honored their voice and interests.

• Academic Growth and Proficiency Through Experiential Learning - In McDowell County, NC, a culture of learner-centered math instruction replaced traditional, compliance-driven pedagogy. Students not only gained confidence but showed measurable growth in math proficiency on course-level and end-of-course assessments. Likewise, in Halifax County’s Empowerment Zone alternative school, PBL led to reduced absenteeism and increased depth of student learning, particularly through interdisciplinary projects grounded in local context and entrepreneurship.

• Social-Emotional Growth and Sense of Belonging - OWL's approach centers empathy as a foundational practice, and students reflect this in their growth. At Bartram Academy, a formerly siloed alternative school in Macon County, student attendance rose from 80% to 85% after a reimagined culture prioritized relationships and support. Graduation rates nearly doubled within two years, demonstrating the power of co-designed learning environments that foster connection, relevance, and purpose.

• Workforce Readiness and Real-World Application - Through OWL’s Profession-Based Learning (ProBL) initiatives and partnerships with local businesses and colleges, students tackled authentic community challenges, strengthening their understanding of career pathways and workforce demands. Students reported frequent use of in-demand skills such as communication, creativity, and design thinking—skills aligned with the future of work and life. In Caldwell County, teacher perceptions of student workforce readiness rose by 73.68% as a result of these initiatives.

• A Culture of Equity, Innovation, and Possibility - OWL’s work helps schools move beyond isolated innovations to system-wide shifts that sustain and scale learner-centered practices. In every district—from urban systems like Albuquerque to rural districts like Halifax and Clay Counties—OWL’s support has helped schools cultivate a culture of innovation where student work is public, interdisciplinary, and deeply connected to community. Seven student showcases in Halifax alone reflected this transformation, making learning visible and relevant.

This evidence affirms OWL’s belief that the most meaningful educational outcomes arise when schools nurture the inherent desire of every student to learn and grow through collaborative, inquiry-driven, and equitable learning environments. The data demonstrates that when schools embrace a culture of co-design and continuous improvement, students thrive—not only academically, but socially, emotionally, and as change-makers in their own communities.

Attachment 2: Open Way Learning Service Impact Methodology

Since its founding in 2018, Open Way Learning has been committed to fostering learner-centered innovation through direct partnerships with schools and districts, as well as professional development workshops for educators. To accurately quantify our reach and impact, as indicated on annual impact reports and on the OWL website, we utilize the following methodology for calculating the number of schools, teachers, and students served. Additionally, we track our pro bono services to ensure transparency in our commitment to equity and accessibility.

1. Schools Served

   • The primary basis for calculating the number of schools served is our official partner/client list, which includes schools and districts with which OWL has engaged in professional development, coaching, or consultation services.

   • Each school or district is counted once per year in the total count, regardless of the number of engagements within that year.

   • For conference-based workshops where teacher cohorts represent multiple schools, we estimate school representation by using participant self-reporting data or regional/national participation norms (e.g., an average of 1.5 schools per teacher cohort).

2. Teachers Served

   • The number of teachers served is based on attendance records from our direct partnerships and professional development sessions.

   • For multi-day coaching or embedded professional development, the number of participating teachers is counted per distinct engagement (not per session).

   • For conference workshops, where participation is not tied to a specific school partnership, we use a standard estimate of 15-25 teachers per session based on historical attendance.

3. Students Impacted

   • Because OWL primarily supports teachers who, in turn, impact students, we use an industry-standard multiplier to estimate student reach.

   • A commonly accepted ratio for professional development impact is 1 teacher = 20-30 students per year (based on average class sizes and teacher-student interactions).

   • For long-term engagements (e.g., multi-year partnerships), the cumulative impact on students is estimated conservatively to avoid over-counting (e.g., assuming an average teacher retention rate within the school system).

4. Tracking Pro Bono Services:

   OWL provides professional development and consulting services at no cost to schools or districts that otherwise lack financial resources. We track this work using the following methodology:

• Direct Pro Bono Services: Schools and districts receiving fully subsidized engagements are included in our overall school and teacher impact calculations, but also categorized separately for transparency.

• Grant-Funded or Sponsored Programs: Initiatives like the WNC Resilience Project, where external funds are raised to eliminate participation barriers (e.g., covering substitutes, travel, or lodging), are documented based on:

  • The number of educators whose costs were subsidized.

  • The percentage of total event attendees receiving financial assistance.

  • The estimated reach of the initiative, using the same teacher-to-student impact ratio.

• In-Kind Contributions: Time spent by OWL staff on pro bono work is recorded in internal logs to track service value in relation to our standard consulting rates.

Attachment 3: From Traditional Metrics to Learner-Centered Process and Outcome Measures

Traditional Metric	Process Measure (Are we doing it well?)	Outcome Measure (Is it producing impact?)
Standardized Test Scores	Engagement in authentic learning experiences	Deeper learning, relevance, knowledge application
Attendance Rates	Agency, voice, participation patterns	Sustained engagement, student ownership
Teacher-Led Assessment	Peer/self/community feedback cycles	Mastery via exhibitions, portfolios, credentials
Seat-Time/Credit Hours	Interdisciplinary access, flexible pathways	Competency + career readiness
Top-Down Policy Implementation	Co-design in learning environments	Systemwide learner-centered practice adoption

Notes for Context:

• Process data ensures we capture how we are implementing our strategies and adapting in real-time.

• Outcome data verifies whether those strategies are leading to the growth and transformations we seek.

• Together, they create a feedback loop for sustainable, resilient innovation.