California's SIG2 Evaluation Highlights

The State Improvement Grant (SIG2, 2005-08) is the three-year continuation of an earlier SIG grant awarded by the federal Office of Special Education Programs (OSEP) to improve special education outcomes and services in California by implementing activities directed at personnel development, family involvement, and systems change.

The partners involved in California's SIG2 effort undertook a broad range of activities starting in February 2005, the majority of which are presented in this report. SIG2 is currently in its third year, having completed the first on January 31, 2006 and the second on January 31, 2007. SIG2 activities include:

The Leadership Community

As part of SIG2, high performing sites are identified through a rigorous, competitive process each year. These sites are awarded for best practices in the areas of literacy, behavior, collaboration, transition and family partnerships. Sites must commit to:

During SIG2, 27 Leadership Sites have been awarded, 23 to schools and 4 to districts and County Offices of Education (COEs). Building upon the SIG1 focus of SE/GE collaboration, the majority of sites have been awarded in the Core Message Area of Collaboration.

Leadership Sites outpaced statewide growth in ELA proficiency. These sites showed an average increase of 10 percentage points for students with disabilities (statewide it was 5 percentage points) and 9 percentage points for all students (statewide was 8).

Average ELA Proficiency at Leadership Sites
27 Leadership Sites awarded during SIG2
  2003-04 2004-05 2005-06
Leadership Sites:
All Students
50% 53% 59%
Statewide:
All Students
37% 42% 45%
Leadership Sites:
Students with Disabilities
14% 18% 24%
Statewide:
Students with Disabilities
15% 17% 20%
AMO Benchmarks (approximate) 12% 23% 23%

During 2005-06, interviews were conducted with the Leadership Sites to learn more about their systems change processes. Out of these interviews, profiles have been developed telling the story of each site's unique experience and approach. These profiles and a cross-site analysis that summarizes what was found entitled, They Are All Our Kids, can be found at: www.calstat.org/leadershipSites/index.html

Statewide Leadership Institutes

The Statewide Leadership Institute was developed to support the development of collaborative systems involving general and special educators and families in implementing effective, research-based educational programs and strategies. Many participants and sites have become leaders in peer coaching and scale-up efforts in the state.

From 2005 to 2007, three annual three-day institute meetings were held with nearly 700 participants attending as part of teams from Leadership Sites and Regional Institute Host Sites. Special and general education teachers were the predominant groups attending the event, along with a strong presence of special and general education administrators, and parents. Approximately two-thirds of the teams have had a parent member attending as part of their team.

The Statewide Leadership Institutes focus on teaching systems change skills along with providing content expertise and giving site teams the opportunity to practice skills and discuss strategies for approaching their unique situations. These are not just events with speakers standing in front of groups, but dynamic days of sharing experiences and strategies among sites and learning to become change agents.

Statewide Institute Participants by Role
2005-07 ~ 698* participants at 3 events
  Number of Participants Percent
Teacher: Special Education 165 25%
Teacher: General Education 145 21%
Administrator: Special Education 87 13%
Administrator: General Education 136 20%
Other Agency Personnel 37 5%
Program Specialists 47 7%
Parent/ Family 61 9%
*20 participants did not specify a role

Online Conferences

Online conferences have been increasingly used by CalSTAT-supported learning communities as a venue for sharing idea and strategies, asking questions, and staying connected with colleagues.

To foster an experience that has greater breadth and depth than a three-day Statewide Institute can provide, participants are asked to engage in an intense wrap-around online conference the week before, the week of the event, and the week after. During the February 2006 institute, 240 postings were made in the online conference. Regional Institutes made 110 postings in 10 topic areas.

Community Network

To facilitate the exchange of knowledge within the learning community, a Community Network was established in February 2006. This is a web-based database where sites can explore the profiles of other Leadership Sites and Regional Institute Hosts to search for upcoming events and identify potential TA providers.

Training and Technical Assistance (TA)

Training and TA are provided around research-based core messages from seven Core Message Areas through:

The primary focus of TA in SIG2 has been on SE/GE Collaboration, with Literacy and Positive Behavioral Supports also being emphasized. The Training and TA provided in the other Core Message areas is also vital to engaging sites in addressing and growing in relation to the many aspects of practice that are critical to educational success for students with disabilities.

TA Events by Core Message Area
2005-07 ~ 275 events with 8,298 participants
  Number of Events Percent
SE/ GE Collaboration 165 59%
Parent Partnerships 10 4%
Transitions 7 3%
IDEA 6 2%
Literacy 53 19%
Least Restrictive Environment 5 2%
Behavior 29 11%

TA Participants by Role
2005-07 ~ 4,926* participants who completed evaluations at 275 events
  Number of Participants Percent
Teacher: Special Education 1,282 28%
Teacher: General Education 1,367 29%
Administrator: Special Education 249 5%
Administrator: General Education 360 8%
Other Certified Professional 587 13%
Paraprofessional 168 4%
Parent/ Family 166 4%
Other 424 9%
*323 participants did not specify a role

Integration with other SIG2/CalSTAT Activities

Many of the teams participating in the BEST, ERIA, and Leadership Site activities also access these TA events, both to reinforce and expand upon the benefits of these programs and to support the systems change necessary to fully implement researchbased practices with fidelity.

Evaluations of TA events from the first two years of SIG2 found that:

Follow-up emails with Training/TA participants several months later found that:

Outcomes at School/District Sites that Have Received 3+ days of Training/TA

Using California Standards Test (CST) state assessment data, ELA proficiency was examined comparing the baseline year of 2003-04 with 2005-06 for 545 school and district sites that have received 3 or more days of TA during SIG2.

Average ELA Proficiency at Leadership Sites
27 Leadership Sites awarded during SIG2
  2003-04 2004-05 2005-06
Leadership Sites:
All Students
43% 48% 50%
Statewide:
All Students
37% 42% 45%
Leadership Sites:
Students with Disabilities
17% 20% 23%
Statewide:
Students with Disabilities
15% 17% 20%
AMO Benchmarks (approximate) 12% 23% 23%

Building Effective Schools Together (BEST)

BEST is a schoolwide discipline program based on positive behavioral supports (PBS) and helps schools develop and implement positive school rules, rule teaching, and positive reinforcement systems. During SIG1, training was done with a cadre of 62 program specialists, administrators, parents, and others from districts and counties throughout the state, who then delivered 37 BEST trainings to approximately 1,600 participants from 197 school site teams in their local districts and areas.

After two years of ongoing cadre training and coaching support during SIG2, 143 of the 197 sites (73%) have implemented BEST/PBS to varying degrees. These include 84 elementary schools, 30 middle schools, 15 high schools, and 14 other schools (charter, continuation, alternative, etc.). In 2006-07 the cadre sites were each given the opportunity and funding to provide an additional BEST training. Five sites received funding and trained approximately 200 participants from 35 new school site teams.

Of the 143 original sites that have implemented BEST/PBS, at least 114 sites (80%) reported having "Partially or Fully Implemented" BEST.

Comparing the baseline year and the most recent year (mostly 2005-06 data) of the partially/fully implementing sites for which data were available, reductions in Office Discipline Referrals (ODRs) and Suspensions were seen at the majority of sites.

Reductions in Referrals and Suspensions at Sites Implementing BEST
  Referrals
n=37
Suspensions
N=43
Reduction > 60% 16.2% 11.6%
Reduction 20% to 60% 32.4% 44.2%
Reduction < 20% 5.4% 11.6%
No Reduction 45.9% 32.6%

Improvements in Student Academic Outcomes at the BEST School Sites

Recognizing that many factors are influencing student academic outcomes at the sites, it is certainly possible that positive changes in the school environment due to BEST/ PBS may have contributed to improvements in student academic performance, indeed numerous sites have attributed their positive changes to having implemented BEST.

An examination of the data for 97 of the partially and fully implementing BEST sites shows positive change in student ELA proficiency at the majority of sites (comparing a baseline year of 2003-04 with 2005-06). Taken as a group, these BEST sites started out with a lower percentage of students ELA proficient than the statewide averages.

Integration of BEST/PBS site teams with other SIG2 activities

18 BEST sites received Collaboration TA, including 10 who receive 3 or more days

16 BEST sites received Literacy TA, including 8 who received 3 or more days

2 BEST sites trained by our Cadre of trainers have become Behavior Leadership Sites

Additional Training Outside of SIG2 by Cadre BEST/PBS Trainers

As some school districts and COEs have seen the benefits of implementing BEST schoolwide positive behavioral supports, they have allocated local funds to support their local Cadre trainers in providing training and ongoing support to over 300 additional school sites, to date.

BEST/PBS participant comments

The wealth of ideas and strategies has been great. It provides a focus for our school community and addresses the issue of school-wide behavior. It supports teamwork and positive behavior recognition. We are thinking, always, about doing and being our BEST! It takes lots of time and energy and patience to implement. We can see why we have been advised to give this process 2 to 3 years! Maintaining staff focus, understanding, participation, and support is a constant challenge, and we have been mostly pleasantly surprised by the response. It allows me, as principal the opportunity to get into classes regularly to give positive recognition for acceptable and model behavior. The importance of teaching and re-teaching and modeling rules has been very powerful in getting things more calm and focused on the playground and other common areas. -Administrator

We have seen a great reduction in the amount of small discipline problems. The students like knowing that everyone in the school is following the same rules and that everyone is enforcing the rules. -Administrator

Effective Reading Intervention Academy (ERIA)

The Effective Reading Intervention Academy (ERIA) supports schools in identifying struggling students and provides them with effective research-based interventions to assist them in improving specific student reading skills. While this activity was not funded under SIG2, it was piloted by CalSTAT and is included in the SPDG application.

Thirty school sites (3 cohorts of 10 schools each) have been trained, involving 240 participants from school site teams including primarily teachers (general and special education), along with administrators and program specialists. Cohort 1 (West Orange County) was trained in 2004-05, Cohort 2 (Southern San Joaquin Valley) in 2005-06 and very recently Cohort 3 (San Diego) in 2006-07. Site teams are receiving ongoing coaching.

Integration of ERIA site teams with SIG2 activities

8 of the 9 Cohort 1 sites received 2 days of SE/GE Collaboration TA

2 of the Cohort 1 sites have become Literacy Leadership Sites

Improvements in Student Academic Outcomes at the ERIA School Sites

Using 2003-04 as the baseline year, the Cohort 1 sites that began implementing ERIA in 2004-05 and continued implementation in 2005-06, show very positive changes in student ELA proficiency.

ERIA Cohort 1 ~ ELA Proficiency
Average of 9 ERIA Sites compared with State Proficiency and AMO Target
  2003-04 2004-05 2005-06
Leadership Sites:
All Students
48% 55% 57%
Statewide:
All Students
37% 42% 45%
Leadership Sites:
Students with Disabilities
13% 17% 27%
Statewide:
Students with Disabilities
15% 17% 20%
AMO Benchmarks (approximate) 12% 23% 23%

An online conference room was created for each ERIA cohort, along with a conference room for the entire community. In these conferences, 329 participants have made 530 postings since August 2006.

Data Tools Have Been Developed

Training Evaluation Database (TED)

TED is a sophisticated Filemaker Pro database programmed to enable sites that provide professional development to track events, participants, and evaluations across different funding sources, and most importantly, to be able to get a series of reports based on that data at the touch of a button.

Through the first two years of SIG2, TED has been installed at 14 sites throughout California: 8 Special Education Local Plan Areas (SELPAs), 1 COE, 3 school districts, and 2 SIG partners--the Diagnostic Center in Southern California and California's Special Education Early Childhood Administrators Project (SEECAP).

Our SELPA has been using TED for several years now. From the beginning, we have found it to be very user-friendly and extremely useful for keeping track of workshop registrations and attendance. One feature that has been particularly nice is the ability to enter workshop evaluations into the program and then generate reports from that information... We love TED! -SELPA Coordinator

CST Charting Program (California Standards Test - CST)

The CST Charting Program was designed to enable school and district sites to create a single page graphical chart showing the percentage of all students compared with students with disabilities as a subgroup performing at the five levels of proficiency on the California Standards Test for English language arts. The chart shows the achievement gap, and that most students have a long way to go in achieving grade-level proficiency. Using the same tables of numbers that parents, teachers, and administrators have had access to for years, the CST Charting Program presents this information with colors and proportions and makes it simple for most people to grasp where students are at a glance.

Nearly 600 copies of a CD-rom (or an emailed version) containing the CST Charting Program have been distributed to personnel and parents. Also included are reporting templates as well as a series of brief instructional movies for how to use the program.

The CST Charting Program is brilliant and invaluable. Simple to use, the program produces an elegant graphic that shows clearly whether we are closing the gap between groups/subgroups. Thanks to ERIA, we are doing just that! -School District Administrator

For more information on TED and the CST Charting Program, go to www.calstat.org/evaluation.html

Family Participation Fund (FPF)

The Family Participation Fund (FPF) was developed to meet the need for fiscal resources to help encourage and support meaningful family/consumer involvement in local, regional, or statewide decision-making activities, events, and groups. Ethnic minorities and low-income families are often underrepresented in the membership of decision-making bodies. The FPF implements outreach activities targeting these underrepresented groups to encourage their participation.

During SIG2, 3,272 stipends were provided to 466 family members (an average of 7 per family over two years) to support their participation in local, regional, or statewide decision-making activities, events, and groups. The FPF successfully reached out to traditionally under-served parents, increasing the percent of stipends going to recipients identifying themselves as ethnic minorities to 83%. The majority of stipends went to recipients who reported that they were African American (53%). The FPF was also successful at reaching out to low-income families: 88% of stipends went to recipients who reported being from a household with income of less than $30,000 annually, and 71% from households with less than $10,000 per year.

FPF Recipients by Ethnicity
2005-07 ~ 3,272 recipients*
  Number of Recipients Percent
African American 1,443 53%
Asian American 45 2%
Hispanic/ Latino 771 28%
Caucasian/ White 468 17%
Other 10 < 1%
*535 recipients did not specify their ethnicity

FPF Recipients by Household Income
2005-07 ~ 3,272 recipients*
  Number of Recipients Percent
Under $10,000 1,957 71%
$10,000 to $19,999 310 11%
$20,000 to $29,999 167 6%
$30,000 to $39,999 114 4%
$40,000 to $49,999 88 3%
Above $50,000 148 5%
*488 recipients did not specify their household income

Interviewed family members reported that within all levels of the education system, they were supporting efforts to keep the system accountable, providing ideas and feedback based on their experiences, and making changes happen.

At this meeting we as parents were welcomed, and our input into policies affecting our children's education was valued and utilized. Our concerns were responded to, and we saw results. -Parent

CAC Capacity Building and Recruitment

As a pilot project in 2005, five Community Advisory Committees (CACs) were partnered with Family Empowerment Centers (FECs) in their area to build the capacity of the CACs as advocates for students with disabilities, to develop support networks for parents, and to increase the membership of underrepresented groups. These CACs include:

It is the mission of all CACs to act as a collaborative partner with the Special Education Local Plan Area (SELPA) to ensure that students with disabilities receive free and appropriate public education and have equal access to all services. CACs serve as a liaison between the SELPA, COE, local districts, families, and community members. Therefore, it is important that the membership of a CAC be representative of its community socioeconomically, ethnically, and by disability.

These CACs have helped parents take a larger role in the decisions affecting their children. This was achieved through a variety of strategies including:

This partnership has allowed the parent advocates of the CAC to benefit from the resources and experience of FEC personnel. Much of this support comes in the form of general assistance with procedure, budgeting, and communication with school officials. Material support has included providing advertising, transportation, and translation services. Specific examples of these contributions include the following:

Additional Activities

TEACHCalifornia.org Website

The Teach California website, a project of the CDE, delivers tools to help potential teachers make the decision to teach, become credentialed, and find job placement. This teacher recruitment resource has been visited more than 800,000 times by over 300,000 people during SIG2.

The TEACH California website also distributes materials to counselors and teachers who offer career guidance to potential teachers. Ordering through the website totalled over 57,000 brochures, 3,000 videos and CDs, and 600 toolkits during SIG2.

The Special EDge Newsletter

The Special EDge is published three times a year and delivered to approximately 50,000 print and email subscribers. It is designed to inform and support parents, educators, and other service providers on special education topics, focusing on research-based practices, legislation, technical support, and current resources. Current and past issues are available online in English and Spanish.

Issues published during SIG2 include: IDEA 2004's New Regulations; Disproportionate Representation; Educational Leadership; Response to Intervention: An Overview; Schoolwide Behavioral Supports at the Secondary Level; and Special Education and High School Reform.

Podcasts

Podcasts are the newest addition to the CalSTAT website. Users can hear recorded conversations with experts from the field on a variety of topics featured in The Special EDge and CalSTAT activities. Eleven podcasts are currently available, with titles including What is Disproportionate Representation? and the multi-part series, Communities of Practice.

The RiSE Library

CalSTAT also manages the RiSE (Resources in Special Education) Library, which makes available documents that offer research-proven strategies and information recommended by educational stakeholders. Residents of California can check out these resources free of charge by ordering through phone, email, U.S. mail, fax, or by visiting the library in person. The RiSE Library is located at Parents Helping Parents of Santa Clara, a federally funded Parent Training and Information Center.

During SIG2, over 2,000 items have been checked out through the RiSE Library.

This activities summary was developed by the CA SIG Evaluation Team: Cheryl "Li" Walter, PhD, Kelly Bucy, MPA, and Alan Wood on behalf of CalSTAT and CDE.

Funds for this project come in part from federal funds awarded as a State Program Improvement Grant to California (CFDA 84.323A) allowed in Part D of Public Law 108-446, the Individuals with Disabilities Education Act (IDEA), as amended in 2004. These dollars are considered local assistance funds and will assist individuals serving children birth to 22 years of age and their families.

Visit CalSTAT online at www.calstat.org

May 2007