California Department of Education, Special Education Division’s special project, California Services for Technical Assistance and Training (CalSTAT) is funded through a contract with the Napa County Office of Education. CalSTAT is partially funded from federal funds, State Grants #H027A080116A. Additional federal funds are provided from a federal competitively awarded State Personnel Development Grant to California (#H323A070011) provided from the U.S. Department of Education Part D of the Individuals with Disabilities Education act (IDEA). Opinions expressed herein are those of the authors and do not necessarily represent the position of the U. S. Department of Education.
The Effective Reading Interventions Academy (ERIA) delivers training and ongoing coaching to school site teams of teachers and administrators, helping schools to improve literacy outcomes for their middle and high school students.
ERIA supports schools in taking the following 5 Steps as an approach to improving instruction and outcomes as they move towards implementing the foundational elements of RtI2 (Response to Instruction and Intervention):
Implementing ERIA's five steps impacts student learning while introducing the concepts and practices of needs-based, rather than labels-based, intervention and education. Over the course of three years of training and coaching, ERIA brings the language, foundational elements, and strategies of RtI2 into the school cultures and faculties of middle and high schools.
Ultimately, ERIA practices are assimilated into the professional cultures of the ERIA sites, with the site teams establishing sufficient expertise and structures to continue supporting improved student outcomes through sustained implementation of ERIA practices without ongoing external supports.
Many veteran ERIA sites have made the internal decision to build upon their work in ERIA, leveraging local resources with ERIA's 5 Steps, to pursue districtwide scale-up of these practices or expand them to fully implement RtI2 in a manner aligned to the National RTI Center’s guidelines (at http://www.rti4success.org).
In December 2009, an evaluation of sites implementing ERIA for at least 3 years observed an average eleven percentage point increase for both all students and students with disabilities in the percent of students scoring proficient or above on statewide English-language Arts testing.
This document reflects the contributions of content experts and program planners who redesigned and updated ERIA in early 2010. These contributors include:
This document also draws from the recommendations and resources generated by:
This document was created for CalSTAT by the SPDG Evaluation Team of Cheryl “Li” Walter, PhD, and Alan Wood.
eria.support@calstat.org | (707) 287-0054
CalSTAT (California Services for Technical Assistance and Training), at Napa County Office of Education, is a special project of the California Department of Education, Special Education Division.
Visit CalSTAT at http://www.calstat.org.
“California Department of Education, Special Education Division’s special project, California Services for Technical Assistance and Training (CalSTAT) is funded through a contract with the Napa County Office of Education. CalSTAT is partially funded from federal funds, State Grants #H027A080116A. Additional federal funds are provided from a federal competitively awarded State Personnel Development Grant to California (#H323A070011) provided from the U.S. Department of Education Part D of the Individuals with Disabilities Education act (IDEA).”
“This project is supported by the U.S. Department of Education, Office of Special Education Programs (OSEP). Opinions expressed herein are those of the authors and do not necessarily represent the position of the U.S. Department of Education.”
The Process of Establishing and Implementing ERIA
ERIA Trainings and Ongoing Coaching Support
A Pathway Towards Needs-Based Intervention and RtI2
Increasing ELA Proficiency at ERIA Sites 16
ERIA’s annual trainings, ongoing technical assistance, and coaching are organized and delivered to schools in regional cohorts. Each cohort is comprised of a group of schools from a limited number of school districts in the same region. Because the schools in the cohort advance through ERIA’s 5 Steps together, attending the same trainings and facing similar challenges, each cohort forms a collaborative learning community, which supports the sustainability and future scale-up of ERIA practices.
Trainings are delivered to a collaborative Site Team, including teachers and administrators, from each school participating in ERIA. The Site Team members guide and support the process of implementation at their schools.
The following schedule describes how ERIA is delivered to and implemented by sites over their three years of participation:
Sites establish the basics of ERIA, including formation of a Site Team, initial training of staff, piloting of assessments and interventions, and planning for full implementation in Year 2.
Sites begin to fully implement ERIA, using data to identify and assess the specific needs of struggling readers and delivering interventions with fidelity. These activities are fully integrated into the school’s master schedule. The Site Team effectively provides leadership and support for schoolwide implementation of ERIA intervention programs and best practices.
Sites are fully implementing ERIA and beginning to focus on sustaining and building upon these practices, including coaching and supporting fidelity of implementation in the classroom, collaborative data review, and the active use of enhanced instructional practices throughout the core curriculum. ERIA practices have become a part of the school culture, and processes are in place so teachers new to the school are trained and mentored.
Four years after the initial training, ERIA sites graduate and must become self-sustaining without outside supports. However, the learning community formed by the ERIA cohort continues to function, and these ERIA graduates often continue learning from and sharing with other regional schools as they further develop best practices in all areas.
Among the supports ERIA sites receive, annual trainings are a central component. ERIA Content Experts deliver training to ERIA Site Teams and other key personnel, providing the skills and technology necessary to implement ERIA’s 5 Steps.
The Two-Days-Plus-One Training establishes the basics of ERIA. The first two days are with a Content Expert, such as Kevin Feldman, Jan Hasbrouck, et al., who addresses an overview of ERIA and RtI2, such as:
The third day is delivered by a different Content Expert who adds to and reinforces this framework. An additional Booster Training is delivered later in the school year to reinforce implementation and address specific challenges Site Teams have encountered.
The Booster Trainings are focused on how to build upon ERIA’s 5 Steps and to maximize their impact on student outcomes, fleshing out a comprehensive strategy to improve student outcomes after the basic framework has already been moved into place. Additionally, because the booster training makes a Content Expert available to Site Teams which are in the process of implementing ERIA, the booster training can address site-specific challenges and opportunities, such as adding interventions to the core curriculum to address widespread needs or establishing intensive supports for those at greatest risk.
Ongoing Coaching and Support is provided by a Cohort Coach, an accessible resource for sites throughout the year to help Site Teams plan, problem-solve, and support schoolwide implementation. In addition to joining the ERIA Content Expert at primary trainings, the Cohort Coach is funded to visit ERIA sites directly, providing coaching and supporting fidelity of implementation in the classroom.
Data Collection and Monitoring is the crux of ERIA implementation, and none of ERIA’s 5 Steps can proceed until a minimal level of data collection and monitoring is established. Schools are supported in this process by the ERIA Site Team, who receive training and coaching support in the use of existing data, making new assessments, and working with that data to inform decision-making and direct resources.
Once data has been collected, the ERIA Site Team will work with the Cohort Coach to establish criteria that directs available resources to support students around their specific needs. Additional support in data collection and monitoring is also delivered in the form of a data collection stipend and analysis from ERIA program evaluators.
ERIA fosters collaborative relationships: to deliver training and support to schools, in regional and districtwide learning communities, and in schools and classrooms.
The Site Team motivates and guides implementation of the 5 Steps, communicates practices and processes to the faculty, and collaborates with other educators and stakeholders to direct the use of interventions and enhanced instruction to improve outcomes both for individual students and the school as a whole. Site Team members receive direct content training and ongoing support in this endevour from external Content Experts and the Cohort Coach.
Because the Site Team is key to ongoing implementation of, not just establishing, ERIA practices, it is essential that new members are actively recruited and supported so they can contribute effectively and develop into leaders. Having attended an ERIA training is not a prerequisite for membership in a Site Team, which benefits from the perspectives and participation of educators from a variety of roles and levels of experience. Developing materials and expertise that allows new members to learn about ERIA practices without external training is key to the sustainability of implementation.
In additional to school Site Teams, ERIA establishes support structures at the statewide and regional level, as well. Statewide, CalSTAT provides leadership and funding, contracting with ERIA Content Experts, Regional Coordinators, and Cohort Coaches. In regional cohorts, the Regional Coordinator and Cohort Coach work with school and district personnel to create and sustain a learning community, which supports both implementation and scale-up efforts.
These cascading structures of support follow a model for establishing implementation of best practices that balances limited resources against the challenges to scale-up efforts in a state as large as California. As a special project of the California Department of Education, Special Education Division, CalSTAT has observed many other successes in creating similar structures across the state to support of a variety of goals, including literacy and PBS.
CalSTAT |
|
|---|---|
| Regional Cohorts Support, Learning Community |
|
| ERIA Site Teams Leadership, Data Interpretation |
|
| Schoolwide Systems Change Implementation of ERIA |
|
ERIA sites deliver needs-based, not labels-based, interventions. Like in RtI2, ERIA sites identify students for additional supports based on assessed student needs and do not group students for intervention based on labels or funding category. ERIA is designed to assist schools in establishing the foundational elements of RtI2, without requiring RtI2’s most resource- and time-intensive aspects.
ERIA’s 5 Steps provide a framework to respond to student literacy skill needs through a comprehensive, collaborative, data-informed process. This is an effective, evidence-based approach to improving the outcomes of students receiving general and special education services, as well as for English-language learners. Research on similar programs demonstrates that:
While ERIA has been implemented in California since 2004, the 2010-11 school year debuts a revision of ERIA which reflects best practices as they are recognized today, with a particular focus on establishing these practices in middle schools and high schools.
ERIA is an ongoing, five step process that addresses the needs of struggling readers and creates a pathway of foundational practices to support more-advanced implementation:
As sites establish implementation of ERIA over the three years of support they receive from CalSTAT, each site will identify opportunities and resources which are unique to their school, as well as challenges which may impact delivery of some of ERIA’s 5 Steps. ERIA accommodates these realities by giving the ERIA Site Team the initiative and autonomy to make decisions and adapt how they implement the 5 Steps to maximize outcomes in a way that is effective for their own school and district.
Many of the following descriptions of the 5 Steps provide basic and advanced implementation options. Sites are encouraged to develop each element towards advanced ERIA implementation, and eventually RtI2, over time.
Struggling readers may have underlying skill needs which, once assessed, can be addressed within an intervention program. The previous year’s CST English-language Arts scores are referenced for each student (or CELDT score) to determine if he or she should be referred for further assessment.
While all students may have specific needs which could be detected through assessment, Site Teams make the decision about which students to further assess based upon available resources.
| CST ELA/CELDT Score | Proficiency Level | Next Step |
|---|---|---|
| 299 or less | Below Basic and Far Below Basic | Definitely refer for further assessment |
| 300 to 349 | Basic | Refer for assessment if resources are available |
| 350 & above | Proficient or Above | Refer for assessment in schoolwide implementation |
CST or CELDT (for English-language learners) proficiency levels are the primary means of identifying at struggling readers. Proficiency levels, generated for all students during STAR testing, are provided by CDE as a scaled score between 150 and 600. A score of 350 or above is defined as “Proficient” or “Advanced,” and scores below 350 are less than proficient. Federal Adequate Yearly Progress (AYP) and Program Improvement (PI) determinations are based on the percentage of students scoring 350 or above.
Students with scores less than “Proficient” (less than 350) have underlying skill needs, and the next question is to use a further assessment to find out which skills need to be further developed through intervention.
Students who are Below Basic or Far Below Basic will definitely need interventions to reach grade-level ELA proficiency, and should all be referred for further assessment.
Students who are Basic may be given effective supports within the core curriculum alone, though sites are strongly encouraged to refer these students for testing as well. Often, as schools increase the number of staff and volunteers who are trained to give assessments, Site Teams will be able to increase the number of students who are referred for assessment, and should be able to meet this standard in Year 3 of ERIA.
In a full RtI2 model, all students are referred for skills assessment. Even for students Proficient and Above, frequent and timely assessment may identify skill needs which can impact outcomes over the course of the school year.
All identified struggling readers are given an oral reading fluency (ORF) assessment to detect whether they have a fluency or decoding word-level skill need. Based on testing, a determination of student needs is made. Interpreting assessment data is a collaborative process, integrating the Site Team and, when available, input from teachers, parents, and even the student.
Reading fluency is a student’s ability to read accurately and quickly, a function of word-level reading skills, and is assessed through an ORF assessment. Commonly, schools implementing ERIA will form an “assessment team” of 3-4 people who can make ORF assessments during brief classroom pull-outs without a substantial interruption of classes. This method can reliably assess around 150 students per week.
Working one-on-one with a teacher, paraprofessional, or volunteer, students read three different grade-level passages (or 8th grade passages for high school students), each for one minute. The person administrating the test makes notes about how many errors were made, how many words were read correctly (WCPM or Words Correct Per Minute), and other notes.
Three examples of evidence-based ORF assessments are:
| ORF Test Outcome | Determination of Specific Skill Needs | Additional Assessment (if resources are available) |
|
|---|---|---|---|
| Errors | Median WCPM | ||
| More than 3 | Less than 70 | Definitely a word-level need (fluency or decoding skills) | Determining decoding or fluency need with a decoding assessment |
| Fewer than 2 | 70 to 120 | Maybe a decoding need Probably a fluency need |
Gather more data: assess for decoding and/or comprehension skills |
| Fewer than 2 | More than 120 | Comprehension needs only | Determine comprehension level with a comprehension assessment |
Using ORF assessments with all students identified as struggling readers provides a substantive basis for guiding the delivery of interventions to specific skill needs, both for struggling readers with and without a word-level skill need. However, schools with more resources may improve outcomes by making additional assessments, to target interventions more precisely to student needs.
It is strongly recommended that schools with sufficient resources conduct a second round of assessments using a new set of tools to further refine understanding of each student’s specific needs. Using the ORF assessment as a guide, struggling readers with a word-level need should receive a decoding assessment, and those without a word-level need should receive a comprehension assessment.
If a struggling reader does have a word-level need and resources are available, an additional decoding assessment can determine whether they need decoding supports or if a fluency-only intervention would be more appropriate.
Decoding is a student’s ability to decipher words and sentences. Students who make many reading errors on a fluency assessment or struggle to decode polysyllabic words are especially likely to have a decoding need.
Ruling out a decoding skill need allows intervention to focus on fluency specifically. Meanwhile, discovering a significant decoding skill need may indicate a need for more intensive, individualized intervention.
Three examples of evidence-based decoding assessments are:
Those who are identified as struggling readers, but for whom a word-level skill need has been ruled out by an oral reading fluency test, will still benefit from intervention. If resources are available, a comprehension assessment can determine comprehension level, to more-effectively direct delivery of comprehension interventions.
Comprehension is a student’s ability to extract useful knowledge from a text and integrates all the elements of the reading process. Even without additional assessments, low CST ELA/CELDT scores suggest a comprehension need which can be addressed directly, once any potential underlying word-level needs have been resolved.
Three examples of evidence-based comprehension assessments are:
Sites implementing ERIA deliver specific interventions to struggling readers based on their assessed needs. Students receiving interventions continue to participate in the core literacy curriculum. Interventions are delivered as an additional service, most commonly through classes within the master schedule or after-school programs.
Individual schools identify and obtain appropriate evidence-based intervention programs based upon the assessed needs of students at their school. Examples of evidence-based intervention programs designed to address specific skill needs are provided below, though sites may implement any evidence-based intervention which has been vetted by the What Works Clearinghouse (http://ies.ed.gov/ncee/wwc/) or Florida Center for Reading Research (http://www.fcrr.org/). Sites receive training, coaching, and other support from the ERIA Content Experts and Cohort Coaches in implementing intervention programs with fidelity.
| Assessed Student Needs | Examples of Intervention Programs |
|---|---|
| Decoding/Fluency (word-level skill needs) | REWARDS |
| Fluency (not decoding) | Read Naturally |
| Comprehension (not decoding/fluency) | REWARDS Plus |
Many evidence-based intervention programs exist for students assessed with word-level skill needs, which may include fluency and/or decoding skill needs. These programs are useful in meeting the intervention needs of students with a variety of skill levels. Some examples of evidence-based decoding/fluency interventions include:
These intervention programs are suitable for students with fluency needs in cases where decoding needs have been ruled out with additional assessment. Two examples of evidence-based fluency interventions include:
Students with a scaled CST or CELDT score of less than Proficient, but who aren’t assessed with a word-level skill need, have comprehension and/or academic vocabulary skill needs which can be addressed with a comprehension intervention. Some schools build these interventions into the core curriculum to address schoolwide comprehension needs, if they exist. Three examples of evidence-based comprehension interventions include:
Many sites deliver interventions within a multi-tiered framework, much like RtI2. In a three-tier model, all students participate in the core curriculum, Tier 1. Students with moderate needs also receive Tier 2 supports (the focus of ERIA), which may include the needs-matched intervention programs described in the above table. Students with the most intensive needs receive individualized, Tier 3 supports.
| Tier 3 Tertiary Level |
Students with intensive needs receive an individualized intervention plan in Tier 3. |
|---|---|
| Tier 2 Secondary Level |
All interventions described so far in this section are Tier 2 interventions (the focus of ERIA) and are delivered in addition to Tier 1 supports. |
| Tier 1 Primary Level |
All students receive supports within Tier 1, the Core Curriculum. Instruction at this level is enhanced with strategies discussed in Step 5 of ERIA. |
To learn more about multi-tiered interventions, browse the National Center on Response to Intervention library, online at http://www.rti4success.org/.
ERIA sites are encouraged to consider delivering individualized, Tier 3 interventions for students who are (1) in the “Far Below Basic” range on CST/CELDT tests and (2) have substantial word-level needs, reading fewer than 70 words correct per minute in fluency assessments. Two examples of evidence-based intervention programs which are appropriate for students with intensive needs include:
Monitoring student progress is a process of assessment and collaborative review, followed by a change in student placements and/or instruction if one is warranted. In ERIA, assessment and monitoring happen at least three times per year, at the beginning of school and again at the end of the Fall and Spring semesters.
Monitoring student progress starts with the collection of new data through reassessment, discussion of the data with teacher input, and a determination of whether the student is properly situated, needs additional supports, or is ready to exit intervention.
| At the Beginning, Middle, and End of the School Year... |
|---|
| Reassess Students in Fluency or Comprehension |
| Collaborative Review Including Student's Teachers |
| Adjust Intervention and/or Instruction if Indicated |
Progress monitoring estimates rates of improvement, identifies whether students are or are not demonstrating adequate progress, and considers different strategies and intervention programs to guide delivery of more effective, individualized instruction. All students who are receiving interventions are assessed with either a fluency or comprehension assessment regularly. Data is charted and used with pre-established targets to determine if individual students are responding to intervention.
The Site Team should consider any changes in student instruction and intervention through a collaborative process informed by re-assessment data. The guidelines and criteria for initial placement used in Step 2 should be considered, and the judgment of the students’ intervention and core curriculum teachers should play a large role in making this determination. Parent and student feedback may also be appropriate.
In fully implemented RtI2, monitoring happens at least monthly, and some ERIA schools have found weekly reassessment to be relatively easy and quite helpful once a robust and computerized assessment infrastructure has been established.
ERIA requires a 3 times per year monitoring schedule that is intentionally less rigorous than the comprehensive progress monitoring of RtI2. However, ERIA sites are encouraged to expand their progress monitoring schedule throughout the year as well as constantly monitoring student progress through the curriculum-based measures that are embedded in the intervention program being used. All of these progress monitoring events present an opportunity to reevaluate intervention and/or strategies as needed to improve student outcomes.
Improving instructional practices and student engagement is an ongoing aspect of ERIA, both in the core curriculum and in interventions. Student success across the core curriculum depends on mastering reading comprehension, a goal which can be best served through schoolwide use of best practices and improved instructional methods.
The Institute of Education Sciences (IES) conducted a research synthesis resulting in five recommendations presented in the practice guide, Improving Adolescent Literacy: Effective Classroom and Intervention Practices, which are supported by ERIA. The guide can be downloaded for free from the IES website, at http://ies.ed.gov/pubsearch/pubsinfo.asp?pubid=WWC20084027.
| The Institute of Education Sciences (IES)'s 5 Recommendations For Improving Adolescent Literacy |
|---|
| 1. Provide explicit vocabulary instruction. |
| 2. Provide direct and explicit comprehension strategy instruction. |
| 3. Provide opportunities for extended discussion of text meaning and interpretation. |
| 4. Increase student motivation and engagement in literacy learning. |
| 5. Make available intensive and individualized interventions for struggling readers that can be provided by trained specialists. |
All five of IES’s recommendations are supported by ERIA, building these strategies into all ERIA trainings and emphasizing their importance for participating stakeholders at all levels. While early ERIA trainings focus mainly on establishing a schoolwide framework (screening, assessment, intervention and monitoring), strategies to enhance instruction become an increasingly important part of ERIA trainings in the second and third years.
Teachers and paraprofessionals, teaching both in the core curriculum and in intervention programs, receive coaching and support throughout the year from the ERIA Site Team and Cohort Coach, in addition to traditional support personnel already in place. These people work with teachers to suggest and guide the use of strategies by educators, enhancing instruction and improving outcomes.
Personnel in these roles can also make direct observations of fidelity of implementation in the school and classroom, a coaching strategy which has been identified as key to achieving successful replication of evidence-based best practices and processes.
Since 2004, CalSTAT has funded training and other supports to assist over 60 sites in implementing ERIA at 23 school districts in 8 counties. In that time, ERIA sites have seen collaboration, the use of interventions, and enhanced instruction help many struggling readers develop English-language Arts (ELA) proficiency. In the chart below, notice the big leap made by each cohort in the year following their start in ERIA.
| 2004 | 2005 | 2006 | 2007 | 2008 | 2009 | |
|---|---|---|---|---|---|---|
| Cohort 1: Sites Starting in 2004 All Students |
45.7 | 54.6 | 56.9 | 55.7 | 56.1 | 60.8 |
| Cohort 2: Sites Starting in 2005 All Students |
40.1 | 43.6 | 53.8 | 50.8 | 51.8 | 59.2 |
| Cohort 3: Sites Starting in 2006 All Students |
35.2 | 36.5 | 37.5 | 43.7 | 46.2 | 49.8 |
| California Statewide All Students |
37.4 | 41.9 | 44.8 | 45.5 | 48.2 | 52.0 |
| Cohort 1: Sites Starting in 2004 Students with Disabilities |
15.9 | 21.3 | 23.4 | 21.8 | 25.0 | 36.8 |
| Cohort 2: Sites Starting in 2005 Students with Disabilities |
4.7 | 15.0 | 10.6 | 16.4 | 20.0 | 22.0 |
| Cohort 3: Sites Starting in 2006 Students with Disabilities |
9.2 | 10.5 | 18.9 | 21.6 | 22.9 | 29.6 |
| California Statewide Students with Disabilities |
14.7 | 17.0 | 19.5 | 20.7 | 24.1 | 30.0 |
| AYP Target (approximate) | 13 | 23 | 23 | 23 | 34 | 45 |
Follow-up surveys and site interviews have been conducted with educators who have attended ERIA trainings, Some of those educators’ comments include the following:
“It has improved all of our small group lessons”
“Money is always helpful... [B]ut, many of the things that make a program like this work can be done without money, just using current resources.”
“We have newer teachers on campus... they can hit the ball out of the park.”
“Due to ERIA’s support, our school was selected to be a site model for the county department of education’s RtI Network. We shared how much the support of ERIA and the systems support helped us make huge gains in our RtI journey.”
“It’s been so drastic that [parents] noticed it and even have made a point to say, ‘Thank you so much for helping them.’ And the students themselves see that they are growing.”
ERIA’s pathway towards RtI2 offers supports and tangible, practical steps schools can take to foster increased ELA proficiency among all of their students.