BEST: Building Effective Schools Together

BEST is a program based on positive behavioral supports (PBS) that helps schools develop and implement positive school rules, rule teaching, and positive reinforcement systems schoolwide. During the 1999-02 State Improvement Grant (SIG1), training was delivered to a cadre of 62 program specialists, administrators, parents, and others from districts and counties throughout the state, who then delivered 37 BEST trainings to approximately 1,600 participants from 197 school site teams in their local areas.

Of the original BEST school sites, at least 143 (73%) implemented BEST to varying degrees. Of these, 54 have continued working with their cadre trainers/coaches into 2007. In 2006-07, 55 new school sites were trained, bringing the total number of currently active BEST school sites to 109.

This report presents available behavioral and CST English-Language Arts proficiency outcomes from the original 143 implementing sites. In addition, this report presents newly-gathered implementation data describing the cohort of 109 current BEST school sites (including 54 continuing and 55 new sites).

Behavioral Outcomes

BEST uses a positive behavioral support approach to reduce behavior problems and create an environment more conducive to learning. To measure these outcomes, office discipline referrals (ODRs) and discipline suspensions were tracked at active BEST sites. Comparing a site’s baseline data (usually 2003-04) with the most recent year available (usually 2005-06), reductions in ODRs and suspensions were seen at the majority of sites with available data.

• 54% of sites showed a reduction in ODRs (major and minor combined), with 49% seeing reductions of 20% or more.
• 67% of sites showed reductions in suspensions (in-school and out-of-school combined), with 56% seeing reductions of 20% or more.

Improvements in Student Academic Outcomes

Recognizing that many factors influence student academic outcomes at the sites, it is possible that positive changes in the school environment due to BEST/PBS may have contributed to improvements in student academic performance. An examination of the data for 114 of the partially and fully implementing school sites (“BEST implementers”) shows positive change in student ELA proficiency at the majority of sites (comparing a baseline year of 2003-04 with 2005-06).

Taken as a group, these BEST sites started out with a lower percentage of students ELA proficient than the statewide averages. While this continued into 2005-06, slight gains were seen for students with disabilities.

• Proficiency for All Students at BEST implementers increased by 6 percentage points.
• Proficiency for Students with Disabilities increased by 5 percentage points.

During the same time period, statewide proficiency levels for all students increased by 8 percentage points (slightly more than BEST implementers), and proficiency levels for students with disabilities increased by 4 percentage points (slightly less than BEST implementers). Compared to these statewide levels of growth:

• 41% of BEST implementers had an increase in the percentage of All Students ELA proficient meeting or exceeding the statewide average improvement.
• 67% of BEST implementers had an increase in the percentage of Students with Disabilities ELA proficient, with 41% of sites showing increases of 5 or more percentage points (meeting or exceeding the statewide average improvement).

Implementation of BEST

Starting in Spring 2007, evaluation of BEST includes use of the PBS Team Implementation Checklist (TIC) developed by Rob Horner and George Sugai to monitor implementation of the program. A total of 72 of BEST’s 109 sites participated in this round of data collection, and many more are expected to complete TICs for Fall 2007. Behavioral outcomes data will become available in Fall 2007, allowing a coordinated comparison of BEST implementers and BEST outcomes.

The TIC asks sites to self-assess fidelity of implementation according to 23 items on two checklists. The first of these lists 17 items relating to start-up activities (staff buy-in, planning, and definition of expectations and policy). The second of these lists 6 items relating to the ongoing activities of mature PBS implementation.

Responses to each item are given as “not started,” “in progress” (partially in place), or “achieved” (fully in place). These responses are interpreted as 0%, 50% and 100% implementation, respectively, for purposes of aggregated analysis.

Full implementation is defined as at least 80% implementation across all survey items. Full implementation of Checklist #1, for example, requires a minimum of 11 items to be fully achieved (roughly two-thirds of items). Partial implementation is defined as average implementation of at least 50% but less than 80%.

• While similar ratios of sites are fully implemented, more continuing sites have advanced beyond minimal implementation (79% compared to 67%).
• Roughly two-thirds of first-year BEST sites have achieved at least 50% implementation of start-up activities.

Implementation by Cadre

BEST training cadres work with sites in one of eight regional areas. Sites within these regional cadres showed varying levels of participation in the Spring 2007 TIC data collection, as well as varying degrees of implementation.

In Spring 2007, 34% of BEST sites did not participate in the TIC data collection. All sites trained by the San Francisco and Santa Barbara cadres (22 total) did not participate, in addition to sites from Napa, Placer, and Sonoma. With more than one-third of BEST sites not reporting, drawing conclusions about implementation of BEST is difficult. These sites are expected to participate in the Fall 2007 data collection, creating a broader picture of the BEST program.

• Every cadre participating in Spring 2007’s data collection included at least one site which indicated full implementation in Spring TICs.
• The Los Angeles cadre included the highest ratio of fully-implementing sites.

Implementation by TIC Checklist Item

The TIC allows sites to document implementation of BEST according to 23 criteria, which are explored on the following pages. TICs completed in Spring 2007 demonstrate that 16 of the 23 TIC items are fully or partially in place as an average across all 72 reporting BEST sites.

Average implementation of these items is summarized by continuing and new BEST sites on the following pages.

• Start-up activities appear well underway, with 64% implementation at continuing sites and 63% implementation at new sites.
• Ongoing activities appear to have stalled at continuing sites, with 27% implementation. New sites showed higher implementation of checklist items (39%), possibly due to more-recent introductory training and engagement in developmental activities.
• TIC items relating to data collection and assessment, both as start-up and ongoing activities, are the least-frequently implemented elements at both continuing and new sites. (See Checklist #1 items 5 and 15, and Checklist #2 items 2 and 4-6.) Additional support in these areas may be required.

Checklist #1: BEST Start-up Activities

• BEST sites show strongest implementation in the areas of Establishing Commitment, Establishing Schoolwide Expectations, and Self-Assessment.
• Administrator support and active involvement appears to be driving the BEST program, with almost program-wide full implementation of this element.
• Continuing sites were less likely to have established a teaching plan for school-wide expectations (item 11, 46%), but were teaching these expectations directly and formally (item 12, 71%). For new sites, this dynamic was reversed (with 56% on item 11, 47% on item 12).
• Continuing sites were more likely to have developed a plan for system support (item 17, 50%) than new sites (27%), suggesting a potential site-to-site mentorship opportunity.

Checklist #2: BEST Ongoing Activities

This checklist was not presented as a required evaluation instrument. Thus, the ongoing activities checklist was completed by only a portion of sites who participated in the Spring 2007 data collection. Of continuing sites, 16 completed Checklist #2. Of new sites, 26 completed Checklist #2.

Checklist #2 refers extensively to the “EBS team,” or effective behavior support team. These site teams lead implementation of PBS at their sites and are more commonly referred to as “site teams” in the context of the BEST program.

• Implementation of monthly site team meeting schedules is most advanced, with 52%.
• With 27% and 39% implementation at continuing and new sites respectively, it appears BEST sites are either not making Checklist #2 items a priority or are in need of additional coaching/support to implement them effectively.

Additional Training Outside of BEST by Cadre BEST/PBS Trainers

As some school districts and COEs have seen the benefits of implementing BEST schoolwide positive behavioral supports, they have allocated local funds to support their local Cadre trainers in providing training and ongoing support to over 300 additional school sites, to date. The majority of these additional sites have been trained by the Los Angeles cadre.

Encouraging these types of district and county scale-ups is a focus of the renewed SIG (which funds BEST), and this has been seen as a great success.

PBS TIC Charting Program

To support sites in using the PBS TIC, the SIG Evaluation Team developed the PBS TIC Charting program. This Excel file includes an electronic version of the TIC to facilitate multi-year record keeping and reporting. Additionally, the program automatically graphs implementation data into each of three separate charts, comparing records from the 2006-07 school year through 2010.

Because the PBS TIC Charting program provides instantaneous graphic feedback to sites, it is hoped that sites trained in PBS outside of the formal BEST program will be able to utilize and benefit from this program as well.

BEST/PBS participant comments

The wealth of ideas and strategies has been great. It provides a focus for our school community and addresses the issue of school-wide behavior. It supports teamwork and positive behavior recognition. We are thinking, always, about doing and being our BEST! It takes lots of time and energy and patience to implement. We can see why we have been advised to give this process 2 to 3 years! Maintaining staff focus, understanding, participation, and support is a constant challenge, and we have been mostly pleasantly surprised by the response. It allows me, as principal the opportunity to get into classes regularly to give positive recognition for acceptable and model behavior. The importance of teaching and re-teaching and modeling rules has been very powerful in getting things more calm and focused on the playground and other common areas. –Administrator

We have seen a great reduction in the amount of small discipline problems. The students like knowing that everyone in the school is following the same rules and that everyone is enforcing the rules. –Administrator