Building Effective Schools Together (BEST)
Summary Report, May 2008

BEST is a program based on positive behavioral supports (PBS) that helps schools develop and implement positive school rules, rule teaching, and positive reinforcement systems schoolwide. To date, over 450 sites and almost 5,000 people have been trained in PBS through the BEST program, including ongoing trainings that have reached over 100 sites and 800 people through the second California State Improvement Grant (SIG2, February 2005 to June 2008), administered by CalSTAT on behalf of the California Department of Education, Special Education Division.

Nearing the end of SIG2, a total of 105 BEST sites remain active. Of these sites, 54 were newly trained in the 2006-07 school year. The other 51 sites were initially trained earlier in SIG2 and have continued to implement PBS and receive booster trainings.

Each BEST school site is served by one of seven BEST regional cadres of trainers which are comprised of local program specialists, administrators, parents and others. Personnel from Institute on Violence and Destructive Behavior (IVDB) at the University of Oregon provided trainings and support to the BEST cadre of trainers and sites in California. These cadres of trainers then provide the initial and follow-up training for school sites as well as assisting each site in implementing BEST. The cadres of trainers are distributed statewide and serve a diverse range of California schools: both rural and urban, coastal and central, in Northern and Southern California.

California's SIG2 BEST/PBS Cadres and Sites

• Los Angeles Cadre (31 sites)
• San Francisco Cadre (18 sites)
• Redding Cadre (14 sites)
• Sonoma Cadre (14 sites)
• Butte Cadre (11 sites)
• Placer Cadre (9 sites)
• Napa Cadre (8 sites)

Implementation of BEST

BEST functions through school-level “site teams”—a collection of administrators and faculty, often including parents and district counselors. These site teams are formally trained in BEST and in turn guide and monitor implementation of the program at their school.

Site teams adapt standard PBS elements and develop new ones as necessary to make sure the program is functional and effective at the local level. This process is driven by site level dialogue and guided by attention to data. Because of the innovations of BEST site teams, there is a great deal of diversity in how BEST is implemented across California.

Common elements of BEST throughout California include:

• Clear and concise behavioral expectations.
  Many sites avoid creating complicated lists of rules for students to follow, instead focusing on formalizing the underlying values from which behavior expectations are derived. The “Three Be’s” are one example which many sites have adopted, consisting of three straightforward guidelines: be safe, be respectful, and be responsible. Many sites modify or add to these standard pieces, such as one site’s fourth Be—”be a leader.” Global, abbreviated standards like the Three Be’s allow behavior lessons to be direct, explicit, and to extend beyond the school.
• Teaching positive behaviors directly.
  BEST sites teach and model expected behaviors directly in a number of different ways. One site produced a series of skits with faculty members acting out positive and negative behaviors, while another held a special “Be Day” event, spending the school day moving students through a circuit of twelve behavior workshops instead of regular classes. Many sites used video cameras in these lessons to allow for subsequent review and reinforcement throughout the year.

“All of the teachers have formulated their classroom rules around the concepts of Be Safe, Be Respectful, and Be Responsible. We have trained the students on cafeteria behavior, playground behavior, assembly behavior, etc. Rules have been explicitly taught and practiced. We have a schedule for reteaching the rules as well. We are off to a great start with BEST!”
–General Education Administrator

• Acknowledging and rewarding positive behaviors.
  Sites often formalize this acknowledgement of positive behaviors by having teachers or staff give out some form of award, ticket or token when they catch a student in the act. These rewards are typically coordinated with a broader theme of good behavior (such as calling them “drops” to reference the book How Full is Your Bucket?), school pride (“good acorn” at a school with “The Mighty Oak” mascot), or another theme emphasizing strong character. These tokens of formal acknowledgement typically serve a double-purpose as raffle tickets or as credits toward prizes, which then further reward the positive behavior.

We implemented the new reward system, and it is making a big difference. The kids love getting them and are trying their best to receive one.
–Parent

“We also have a ‘Caught Being Good’ program, and the kids get tags to hang from their backpack. They can also add perfect attendance tags and academic achievement tags.”
–General Education Teacher

• Attention to the reasons why.
  Many sites participating in BEST approach both teaching positive behaviors and correcting negative ones with a strong focus on the underlying reasons. Conscious and explicit attention to behavior expectations ensure that staff can answer students’ questions without resorting to “I told you so.” Attention to the reasons behind bad behavior is viewed as important as well. One site recognized that their students like and feed “drama,” turning small problems into “passion plays;” they responded by making a rule that staff spend at least two minutes talking to a student as part of any correction, both giving the student a chance to explain the problem and to learn perspective and proportion. Another site’s staff adopted the practice of adding a “because” to every correction (“please pick up your trash so the playground will be clean,” for example), defusing potential confrontations by helping students recognize the reasons for rules instead of just enforcing the rules.

It has been helpful to have a common language that all staff members can use with the students!
–General Education Teacher

• Attention to data and continual reassessment.
  In addition to biannual implementation data and annual behavioral incidence data collected with the support of a CalSTAT data stipend, BEST site teams engage in ongoing self-assessment to build a more effective and responsive PBS program. In one district including a large number of BEST schools, an internal student database was modified to track behavior data alongside academic data, specifically to help BEST site teams. Less formally, another site developed a rubric to guide responses to specific behaviors, only to quickly realize that the behavior problems they were having were completely different than what they expected. The rubric was then updated to reflect these revelations.

Starting in Spring 2007, evaluation of BEST has included use of the PBS Team Implementation Checklist (TIC) developed by Rob Horner and George Sugai to monitor implementation of the program. This instrument asks sites to self-assess degree of implementation according to 17 items relating to activities such as staff buy-in, defining expectations, and teaching behavior expectations.

Responses to each item are given as “achieved,” “in progress” or “not started,” and interpreted as 100%, 50% and 0% implementation, respectively, for the purposes of aggregated analysis.

If all 17 checklist items average to 80% or more, a site is considered to be fully implementing BEST. Most sites averaged at least 50% but less than 80%, and are considered to be partially implementing. Below 50%, sites are considered to be minimally implementing.

• The degree of implementation is advancing at BEST sites. Comparing Spring to Fall 2007, the number of sites fully implementing nearly doubled, from 15 sites to 27.
• Additionally, 11 of the 19 sites who reported minimal implementation in Spring 2007 increased implementation at their sites enough to move to the partial category in the subsequent Fall. (This may not be readily apparent in the pie charts because 6 of the sites who had “not reported” data in Spring 2007 moved to “minimal implementation” in Fall 2007.)
• By Fall 2007, 66 of the 105 (63%) sites reported having partially or fully implemented BEST.

High staff transiency was described as a major hurdle to implementation in some areas, dissolving newly-created site teams year after year. Having time to meet and plan for implementation was another significant hurdle reported.

Implementation by Cadre

BEST is organized around the seven regional trainer cadres presented on page one. Sites within these regional cadres showed varying levels of participation in the Fall 2007 TIC data collection, as well as varying degrees of implementation.

• Four cadres, all with full or nearly-full data reporting, demonstrated high levels of implementation among their sites. These were San Francisco, Los Angeles, Butte, and Redding.
• A total of 25 sites (mostly from Napa, Sonoma, and Placer cadres) did not submit TICs in Fall 2007. Most sites who did not provide a TIC also did not report behavior incidence data (page 9), making it difficult to gauge outcomes in these cadres.

Implementation of Individual BEST Elements

The chart on page seven shows the average degrees of implementation for each of the 17 TIC checklist items by all sites who completed a TIC in Fall 2007. Average responses from Spring 2007 are included as well for comparison purposes (as the gray shadow columns). Of the 17 items, 14 showed increases in implementation.

Items in the TIC are grouped into six categories which relate to different aspects of the BEST program. These and the individual items are explored here and on the next page as averages across all BEST sites. BEST empowers site teams to make implementation decisions at the site level, resulting in a diversity of approaches, so it is important to recognize that an individual school may deviate from these averages.

Exploring average implementation of these items individually, BEST sites overall have demonstrated their commitment and active use of schoolwide positive behavior supports. These elements are seen mostly in the TIC categories of establishing commitment, establishing and maintaining the site team, and establishing school-wide expectations.

• Three items that were fully in place were administrator support and active involvement (item 1), establishment of a representative team (item 3), and defining three to five behavior expectations (item 9).
• Two other items were almost fully implemented, creating a system to acknowledge and reward expectations (item 13) and identifying and involving personnel with behavioral expertise (item 16).

Strong responses to these five items in the TIC demonstrate key aspects of the BEST program overall. Site teams, with support from administrators and staff with relevant expertise, appear to be in place in most BEST sites. BEST sites are also widely implementing use of a concise list of behavioral expectations which they reinforce with a formal reward system. Both of these latter two elements are described on pages two and three of this report.

Meanwhile, the use and integration of data in day-to-day implementation of BEST has seen more moderate progress across BEST sites overall, despite strong implementation and successes in many individual sites. These checklist items are seen primarily in the categories of self-assessment and establishing an information system.

• A number of sites have been slow to complete an audit for efficient integration of the BEST team with other behavioral teams and initiatives (item 5). Note that the formal audit described in this checklist item does not preclude the use of data in less formal contexts.

Finally, many sites have also been slow to develop a plan to identify and establish systems for teacher support, functional assessment, and development and implementation of a support plan (item 17). It appears that many BEST sites would benefit from continued support in cultivating behavioral expertise and fostering the ongoing development of their programs through time.

Stronger Communities, Safer Schools

The most pervasive outcome reported by the teachers, administrators, and parents involved was a cultural shift at schools implementing BEST. By emphasizing the teaching and support of positive behaviors among students, discipline became a proactive enterprise at many BEST sites, with faculty working with students to create a safer and more congenial learning environment.

BEST faculty members related many situations in which students took initiative in promoting this better environment, several of which have been described earlier in this report.

In another site, the Three Be’s were rebranded as “Cougar Pride,” after the school mascot. The site rewards good students by recruiting them to read morning PA announcements. These students adopted, without prompting, the practice of reciting the Cougar Pride tenets every morning at the conclusion of the morning announcements.

Another community-building effort enlisted faculty and staff in a “valet” service each morning, helping students out of cars and with their backpacks. By greeting students and parents each morning, the school created a safer and more friendly environment, starting off every school day on a high note.

One site has created a “village” environment by setting up “special friend” relationships between students and faulty members. These staff-student pairings are created so that faculty members (who intentionally do not have a professional relationship with the student) take an interest in the student, checking up on them and getting to know them. This creates a “trickle” effect where students see teachers caring outside of their roles as teachers, building a community environment.

Many sites have also found ways of enlisting student support in modeling and rewarding positive behaviors. For example, one school recruits students from third and fourth grade to serve as “play leaders” for first and second graders. These leaders apply for the position not unlike applying for a job, receive training and serve roles similar to faculty members in modeling and rewarding positive behaviors. Play leaders become part of the playground and classroom communities of these younger students, learning their names and working with them over time.

“We have started a BEST Club, where students help with how to follow the rules. They also run a table once a week that give the kids a chance to give drops [rewards] to students and faculty.”
–General Education Teacher

Measurable, Behavior-related Outcomes

Changes in the use and application of office discipline referrals (ODRs) varied considerably, with about half of the sites decreasing their use of ODRs, while the other BEST sites saw increases, including 39% whose use of ODRs doubled or more.

These measures are not based on negative behaviors themselves, but on the response of school officials to negative behavior. Increases may be due to changing school policies and practices, such as more consistent enforcement of school rules or enhanced awareness of negative behavior and its impact on a school’s academic mission. Consequently, the increase or decrease of ODR (or suspension) rates at a given site may or may not be evidence of increased incidence of negative behaviors. Clearly, data collected with these measures suggest a wide range of outcomes.

One of the measures showing the largest decreases was of the rate of students receiving more than a single day of suspension over the school year. For 44% of sites who reported a baseline for this statistic, this rate decreased by more than half. This indicates a sharp decline in the number of students suspended for major infractions or with continued, serious behavior problems.

The PBS program taught by BEST utilizes an RTI (response to intervention) approach. In RTI, students are identified for one of three levels of support according to need (support for all students, for at-risk students, or for high-risk students). Students receiving suspensions would be identified for additional support in this RTI-based behavior model, which may be contributing to the decreases seen in students receiving more than one day of suspension at these BEST sites.

The rate of suspensions and ODRs at fully and partially implementing BEST sites for whom baseline data was available are presented on the following page. These sites represent considerable diversity both in student populations and in how they chose to pursue a schoolwide PBS program. These 38 sites suggest the following:

• Almost two-thirds (25 of 38) of BEST sites had a suspension rate below the statewide rate in the 2006-07 school year.
• Over 60% (23 of 38) saw either a reduction in suspension rate or increases smaller than those of California as a whole.
• Some sites saw decreases in suspensions but an increase in ODRs (and a few saw the reverse). This may be an example of how changing school policies and practices can impact statistics in seemingly contradictory ways.

It is important to note that these rates can change for a number of reasons. Changes in school policy, which many BEST site teams chose to implement, could have had a large impact. Examples of this include greater reliance on non-suspension responses to misbehavior or expanding use of ODRs to respond to a wider spectrum of negative behaviors. Increased awareness of behavior issues and attention to accurate record keeping may also be responsible for increases or decreases in these statistics.

Many sites described positive outcomes as a response to increased attention to ODR and suspension data. Indicative stories include the following:

• One site team realized that African American and Latino students were receiving far more ODRs and suspensions than Caucasians. The revelation resulted in a number of uncomfortable faculty conferences, but the process has prompted this site to begin addressing previously unrecognized disproportionality.
• In another site, analysis linked a large portion of their office discipline referrals to the recess after lunch. By moving lunch from before to after the midday recess, many of these behavioral problems (and referrals) were eliminated.

One cadre of trainers has used teleconferencing software to address data collection and interpretation, having all site teams complete surveys and compile data at the same time. This has resulted in 100% data collection for CalSTAT’s purposes, as well as helping sites share information and reinforce each other’s efforts.

Establishing a system for effectively recording, tracking, and accessing data at the site level was noted as a challenge for many sites. Many sites have used a service called SWIS but expressed concern about its cost. Development of less expensive (or free), usable and accessible data systems could be helpful for BEST sites and other schools which are already trying to track and access data for decision-making.

Helping us think about our data collection process was very helpful. We are well on our way and have incorporated it into our student information system.
–General Education Teacher

Ongoing Support for Positive Behavior

BEST’s successes are attributable to the partnerships and teamwork of the agencies and personnel working together to build safer, more productive schools. A theme of partnerships is pervasive in all levels of the program, with coordinated efforts between CalSTAT, the Institute on Violence and Destructive Behavior (IVDB), and schools and districts throughout California. Nowhere have these collaborations been more successful than between the cadres of trainers, school faculty and administrators, parents, and even students.

The positive outcomes of BEST are expected to continue for many schools through the ongoing work of many of these partners and with two additional sources of support.

CalSTAT administers many other education resources, and these have been open to BEST sites who have been interested in pursuing them. Interactions between BEST sites and CalSTAT’s Leadership Site Award program have been particularly productive.

• Paradise Unified School District received a Leadership Site Award for its program of GE/SE collaboration, providing demonstrations and trainings to other schools in the area. Many of Paradise USD’s schools sought out PBS training as well and became BEST sites, to combine the benefits of these different programs.
• Further, two BEST sites went on to receive their own awards as Leadership Sites in the core message area of behavior. These schools, Bidwell Junior High (Chico USD) and Walter Reed Middle (Los Angeles USD) provided additional PBS trainings to neighboring schools with funding from the Leadership Site Award Program.

In Los Angeles Unified School District, the school board recently approved a district-wide PBS program by passing the Discipline Foundation Policy in March 2007.

BEST sites in the district modeled many of the policies that will be advanced under this policy and will continue to receive PBS program support from LAUSD. Additionally, members of the Los Angeles cadre of trainers have become key members in the policy’s Implementation Task Force.

“Every student, pre-school through adult, has the right to be educated in a safe, respectful and welcoming environment. Every educator has the right to teach in an atmosphere free from disruption and obstacles that impede learning. This will be achieved through the adoption and implementation of a consistent school-wide positive behavior support and discipline plan for every school in LAUSD.”
Discipline Foundation Policy
Policy Bulletin, March 27, 2007

This report was developed for CalSTAT by the SIG Evaluation Team of Cheryl “Li” Walter, PhD, and Alan Wood.

CalSTAT (California Services for Technical Assistance and Training), at Napa County Office of Education, is a special project of the California Department of Education, Special Education Division. Visit CalSTAT at http://www.calstat.org.

“CalSTAT is partially funded from federal funds awarded in Part B of Public Law 108-446, the Individuals with Disabilities Education Act (IDEA), as amended in 2004. Additional federal funds are provided from a federal competitively awarded State Program Improvement Grant to California (CFDA 84.323A) allowed in Part D of Public Law 108-447, the Individuals with Disabilities Education Act (IDEA), as amended in 2004. These dollars are considered local assistance funds. Both funds are to assist individuals serving children birth to 22 years of age and their families.”