California’s State Personnel Development Grant (SPDG/SIG3)
Overview

On September 1, 2007, California was awarded a five-year federal Office of Special Education Projects (OSEP) State Personnel Development Grant. SPDG is the new name for the SIG or State Improvement Grant. California is currently completing its second SIG, and will begin implementing the new SPDG/SIG3 in summer 2008.

Purpose: The central purpose of the State Personnel Development Grant (SPDG) is to continue to reform and improve California’s system of personnel preparation and professional development to increase student achievement and improve outcomes for students with disabilities.

Project Activities: Specific areas of activity supported under the SPDG will include:

• Professional development, training and technical assistance in scientific or evidencebased interventions in literacy (Effective Reading Intervention Academy or ERIA);
• The Leadership Community (State Leadership Institute, Regional Leadership Institutes, and Leadership Site Award Program activities) characterized by collaboration between general education and special education; and knowledge and skills gains in the evidence-based CDE-approved “core message” areas;
• Centrally coordinated TA provided by project consultants and site-to-site TA provided by the Leadership Community centered around the evidence-based “core message” areas;
• Data tools for data-informed decision-making at the state and local levels;
• The Family Participation Fund and Parent Training and Information Centers parent/family outreach and development of the Parent-School Involvement Facilitation Survey;
• Expansion of the capacity of the California State University Los Angeles (CSULA) education specialist credential intern program; and
• Improvement of the TeachCalifornia.org website to help recruit special education (SE) teachers.

Project Evaluation: This SPDG Evaluation Overview is designed to familiarize California’s collaborative partners with the coming efforts. Included in this overview, on the pages that follow, are:

• CA’s SPDG/SIG3 Logic Model
• CA’s SPDG/SIG3 Objectives
• CA’s SPDG/SIG3 Evaluation Methods Table

California's SPDG/SIG3 Logic Model

Needs	Activities/Inputs	Short-Term Outcomes	Intermediate Outcomes	Long-Term Outcomes
Increase Reading/English Language Arts Proficiency	Personnel development T/TA in evidence-based instructional practices	Increased knowledge and skill of personnel in delivering evidence-based interventions	Increasing #s of teachers, schools, and districts implementing evidence-based practices with fidelity	Improved Outcomes for Children with Disabilities
SE/GE Collaboration for Systems Change	Leadership Community of Practice	Organizational systems change to support and sustain new practices	Knowledge strategy for learning from experience and sharing with others
Facilitate Family-School Involvement	Stipends to low-income and under-represented families	Increased participation of families at the school, district & statewide levels	Families influencing ed system, becoming more knowledgable about services and better advocates for kids	Improved System of Personnel Development
Recruit and Retain Highly Qualified SE Teachers	Expansion of CSULA Intern Program capacity	Increased enrollment in Ed Specialist Internship program	New highly qualified teachers still teaching 3 years later

CA’s SPDG/SIG3

Objectives Goal: The purpose of CA’s State Personnel Development Grant (SPDG) effort is to continue to reform and improve California’s system of personnel preparation and professional development to increase student achievement and improve outcomes for students with disabilities.

Outcome 1: SPDG activities are aligned with SPP improvement strategies

Objective 1: To provide personnel development Training/TA that is aligned with improvement strategies in the California State Performance Plan (SPP), as demonstrated by:

• Performance Measure 1a: To deliver SPDG activities that are named as SPP improvement strategies in each of the three SPP Improvement activity areas: 1) monitoring and stakeholder meetings, 2) T/TA, and 3) utilizing data in decision-making.

Outcome 2: The percentage of personnel receiving professional development in scientific or evidence-based instructional practices is increasing or scaling up within districts and statewide

Objective 2: To increase the percentage of personnel receiving professional development in scientific/evidence-based instructional practices by scaling up successful SIG2 activities, as demonstrated by:

• Performance Measure 2a: ERIA training will be provided to an additional 30 school sites;
• Performance Measure 2b: Increasing scientific or evidence-based core message T/TA provided by 25 Regional Leadership host sites and 53 Leadership Sites; and
• Performance Measure 2c: At least 80% of professional development T/TA will be based on scientific/evidence-based instructional/behavioral and systems change GE/SE collaboration practices.

Outcome 3: Professional development activities are improving the knowledge and skills of personnel in delivering scientific or evidence-based instructional and behavioral practices

Objective 3: To provide professional development activities/training based on scientific/evidence-based practices, specifically ERIA (Effective Reading Intervention Academy), improving personnel knowledge and skills, as demonstrated by:

• Performance Measure 3a: Increasing the knowledge/skills of ERIA instructional/behavioral practices among training participants by at least 35%;
• Performance Measure 3b: Increasing the knowledge/skills of personnel in delivering instructional/behavioral practices as measured by at least 50% of ERIA training participants repeatedly implementing what they have learned; and
• Performance Measure 3c: Increasing to 75% the sites implementing ERIA in which personnel are observed implementing learned practices with fidelity.

Outcome 4: Personnel knowledge and skills in scientific or evidence-based practices are supported and sustained through ongoing comprehensive practices

Objective 4: To implement strategies and practices that sustain the knowledge and skills of personnel in the implementation of scientific/evidence-based practices, as demonstrated by:

• Performance Measure 4a: Personnel from at least 80% of the schools implementing ERIA will access 3 or more sustaining activities after 2 years in the program;
• Performance Measure 4b: At least 95% of professional development training activities in ERIA will be sustained through ongoing and comprehensive practices; 􀂾 Performance Measure 4c: At least 60% of school sites accessing supporting and sustaining activities will implement ERIA with fidelity within three years;
• Performance Measure 4d: The percentage of all students and students with disabilities who score proficient or advanced on the California Standards Test will increase by at least 5 percentage points in at least 70% of ERIA school sites implementing with fidelity;
• Performance Measure 4e: The project will develop and/or disseminate at least 2 tools or instruments for supporting and sustaining improved practice or system change.

Outcome 5: Parents are involved with schools and the broader educational system

Objective 5: To facilitate parent/family involvement with schools and the educational system as a means of improving services/results for children with disabilities, as demonstrated by:

• Performance Measure 5a: 70% of Family Participation Fund stipends will go to parent/families reporting annual household incomes of less than $30,000; and
• Performance Measure 5b: At least 70% of school site teams attending the State Leadership Institute will include the participation of parents/family members.

Outcome 6: Special Education teacher recruitment and retention is improving

Objective 6: To improve the recruitment/retention of quality SE teachers as demonstrated by:

• Performance Measure 6a: The number of total visitors to the teachcalifornia.org website will increase 10% per year after outreach in year one;
• Performance Measure 6b: The education specialist candidate enrollment in the CSULA Special Education Intern Program will increase by at least 25%; and
• Performance Measure 6c: 90% of CSULA education specialist intern program graduates will be employed in public schools for at least three years after program completion.

CA SPDG/SIG3 Evaluation Methods Table

SDPG Activities	Evaluation Instruments/Data Sources/Methods	Performance Measures
1) All professional development T/TA including: ERIA, Regional Institutes and Follow-up TA, and Leadership Site TA	Training Evaluation Database (TED) Event Core Message Topic Tracking Team and Individual Sign-in Sheets “How Was The Training?” event evaluation survey Online Follow-up evaluation survey	1a, 2abc, 3ab 2abc, 4b 2ab, 4a 3a 3b
2) ERIA	Team Implementation Checklists (TIC) Site Action Plan (SAP) System-wide Evaluation Tools (SET) CA Standards Test Scores (CST) in ELA CST Charting Program Site Academic Measures Metafile (SAMM) ERIA Student Data Tracking Tool Online Evaluation of Coaches	2a, 3c, 4c 2a, 3c, 4c 2a, 3c, 4c 4d 1a, 4d 1a, 4d 4d 4a
3) Leadership Community	Knowledge synthesizing documents Site Academic Measures Metafile (SAMM) Site Data Profiles	4e 1a 1a
4) Parent/Family Involvement	Family Participation Fund (FPF) Invoices FPF After-event Evaluations State Leadership Institute Team Registrations ISES Task Force Roster and Agenda Parent-School Involvement Facilitation Survey	5a 5a 5b 1a 1a
5) SE Teacher Recruitment, Preparation, and Retention	Web Use Tracking Software Online Registered Users Database CSULA enrollment figures CSULA graduate employment retention follow-up	6a 6a 6b 6c

Improving the Special Education System in California
A State Personnel Development Grants Program

presented by

The California Department of Education and the Improving Special Education Services partners

Narrative

Introduction

            The original State Improvement Grant (SIG), as delineated in the 1997 Reauthorization of the Individuals with Disabilities Education Act (IDEA), provided the California Department of Education (CDE) with a unique opportunity to forge broad-scale, collaborative partnerships to improve outcomes for children with disabilities. CDE changed the system of stakeholder input, opening the planning and resource allocation process to a highly representative and inclusive stakeholder group in compliance with IDEA Section 1452(b). The original grant, followed by the SIG Supplement and SIG2, enhanced California’s vision of a special education system of services to meet and align with California’s state performance goals. California is eager to continue this work. Through grant activities that promote general and special education collaboration, a vision of “they are all our kids” is becoming a reality in grant funded sites.

California’s size and multi-cultural population render State Personnel Development Grant (SPDG) resources as vital. California is the most populous state with over 35 million people. In the last decade, between 1990 and 2000, California’s population increased by 13.8%, and by 2025 the state’s population is projected to reach 49.3 million, an increase of over 40%. California has over 6 million enrolled school children—1.7 million in Los Angeles county alone—9,553 public schools and 1,054 school districts. In 2005-06, 683,178 students received special education services, a 5% increase over the numbers just 5 years ago.

Despite the size of California’s economy (now the 5th largest in the world) and its resources, California faces tremendous and growing challenges in public funding for healthcare, housing, transportation, criminal justice/corrections, and education. Taking education alone, California faces challenges that compete with its desire to address system improvement for individuals with disabilities. For example, once first in the world, over the last 20 years California has dropped dramatically in educational attainment. California is now 6 percentage points below the national average in educational attainment as measured by the percentage of 24-35 year olds with an associate degree or higher (Jones, 2006). According to the National Education Association Rankings of the States in 2007, one fourth of California’s public school students have not yet mastered English. In 2005-06, over 1.5 million students or ~25 % of all students were considered English Learners with the highest percentage in the early grades (36% of K-3 children) (Data retrieved from http://dq/cde.ca.gov/dataquest.) In addition, 55% of California’s public school buildings are over 30 years old. The state faces the prospect of investing billions of dollars to repair, maintain, and modernize these aging facilities. (See, http://www.edsource.org/pub_edfct_pred.cfm.)

In light of all these challenges, the state approached planning for this SPDG, formerly called the SIG, with a clear path ahead. The previous and natural alignment of SIG2 activities to the State Performance Plan (SPP) for Individuals with Disabilities facilitated the identification of program components that further California’s SPP goals. With broad stakeholder involvement and support, the state reaped two enormous benefits from earlier SIG programs: 1) evidence about what works and what doesn’t; and 2) as discussed below, an unprecedented consensus of what is needed to further improve special education services in California. Based on these findings, CDE and the stakeholder planning group are pleased to offer this proposal for the next stage of systems change in California.

(a) Need for Project

a.1. Process for identifying gaps or weaknesses in services, infrastructure, or opportunities.  Previously, CDE relied upon several planning groups to address personnel development, assessments, monitoring and accountability, and resource allocation in special education. The California Partnership Committee on Special Education (PCSE) was established by the first SIG program as a large, comprehensive, and diverse stakeholder advisory group pursuant to IDEA 2004 requirements. The Key Performance Indicator Stakeholder Committee (KPISC), another comprehensive stakeholder group, provided input to the development of California’s SPP and more recently focused on establishing evaluative measures for California’s SPP indicators.  In 2006, CDE combined the two stakeholder groups (the PCSE and the KPISC) into one large, diverse stakeholder group of 100+ participants comprised of parents, advocates, special education staff, Program Improvement contractors, professional organizations, and administrator groups. Now, the SPDG advisory group and the state’s SPP and Annual Performance Report (APR) stakeholder groups are combined. The group was recently renamed the Improving Special Education Services: Through Planning, Personnel Development, and Performance Reporting Group, or ISES.  The chart below illustrates the diverse and comprehensive nature of the 2006 ISES stakeholder group attending the most recent meeting in January 2007. The types of agencies and organizations represented are roughly grouped, with the numbers of individuals in attendance through the PCSE since 2002 noted; the letters of commitment from each of these organizations are located in the Appendix.

 

ISES* Role - Partners	2002	2003	2004	2005	2006-2007
Parents and Parent Organizations	35	21	16	20	26
Department of Developmental Services	1	1	3	1	6
CA State Advisory Commission on SE	2	2	2	3	2
Interagency Coordinating Council (Part C) & Part C/B T/TA providers	2	2	0	0	4
National T/TA Providers	2	2	2	2	4
Institutions of Higher Education	12	7	16	12	31
Governor's Office	1	1	1	1	0
General Educators	8	6	2	1	5
Special Educators	45	27	48	45	10
Early Intervention Programs	5	4	3	3	5
Community-based/Non-profit Agencies	19	12	13	12	13
Department of Rehabilitation	2	2	2	2	3
Department of Health Services	1	1	1	1	1
Department of Mental Health	3	3	3	3	1
Other Agencies	46	21	10	9	10

*ISES=Improving Special Education Services stakeholder group, formerly the PCSE (Partnership Committee on Special Education)

With its predecessor groups, ISES has been analyzing the need for systemic change of the special education (SE) system, the priorities for resource allocation to meet these needs, and the results of the SIG/SPDG and other efforts to make the needed changes for almost 8 years.  In the course of its work advising CDE, ISES examines and considers a wide range of data drawn from sources including the California SPP and its APR, statewide assessments, the 618 report Table 6, analyses of individual and district data from the California Special Education Management Information System (CASEMIS), and improvement activity and outcomes reports from professional development training and technical assistance (T/TA) providers, such as SIG/SPDG, Riverside County Achievement Team Training, and the Pre-school Desired Results: Developmental Profile program, and common areas of statewide noncompliance findings gathered from general supervision data (e.g. complaints, due process, program monitoring). This is truly data-driven decision-making in action!

The ISES stakeholder group meets bi-annually to assess system change, review the current SPDG evaluation results, and look closely at improvement activities and where SPP indicator targets are not being met.  It is currently divided into 10 active workgroups organized around SPP key indicators. At the most recent meeting in January 2007, CDE administrators presented on California’s APR actuals vs. targets. After examining the “state of the state,” the group’s workgroups each formulated “next-steps” for improvement activities and at the end of the meeting each produced a set of written recommendations. These recommendations and the priorities guide the staff of the CDE, Special Education Division, in their allocation of resources, including the potential resources requested in this SPDG application.

a.2. Nature/magnitude of weaknesses in services, infrastructure, or opportunities.

Educational Benefit.  For the last seven years, CDE has increasingly emphasized monitoring for educational benefit. This occurred in two ways. First, CDE customized reviews based on district performance on Key Performance Indicators (KPIs) related to the State Performance Plan (SPP) indicators and based on concerns expressed by parents in the district. The items reviewed are selected (via software) to assess procedural issues that are programmatically linked to the KPIs.

The second method for monitoring educational benefit is an educational benefit file review process.  Conducted jointly by CDE and district staff, the educational benefit review involves a detailed look at the IEPs planned for a child over 3 years old. As a part of the review, the team evaluates the relationships among present performance, identified needs, goals and services established to support needs, and progress in each need area.  In addition, IEPs are compared across years to determine if the IEPs were reasonably planned to result in educational benefit (i.e., Were goals and services adjusted in response to student progress?).  As reported in the SPP General Supervision, Indicator 15, these processes identified increased monitoring findings in:

• provision of free and appropriate public education (FAPE);
• access to and progress in the general education curriculum;
• qualifications of personnel;
• individualized education program (IEP) development;
• curriculum modifications and supports for school personnel; and
• assessment.

These results reflect several general problems in providing inservice and support to staff and planning services for students: 

• SE teachers may not have adequate preparation in the core academic subjects they teach;
• SE staff may not be included in adoption/training related to core curriculum/textbooks, resulting in inadequate preparation;
• SE staff may not have adopted curriculum and instructional materials aligned to the state standards;
• IEPs for students with severe disabilities contain few goals or objectives related to access to, and progress in the general education curriculum;
• Emphasis on standards-based IEPs (particularly for students with specific learning disabilities) led to lack of planning for needs related to disability (sensory and motor supports, fine motor skills, etc.); and
• Instructional supports for English language development and core curriculum areas inadequately reflect IEP planning processes for students who are English Learners.

These monitoring findings account, in some measure, for the statewide results in the SPP and Annual Performance Report for Indicators: #3-Statewide Assessment; #4-Suspension and Expulsion; and #5-Least Restrictive Environment, as discussed below.

Special education students’ participation in Standardized Testing and Reporting (STAR) statewide assessments. STAR combines scores from various tests, depending on the grade, disability status, and English Learner status, but the primary assessment is the grade/subject specific, criterion-referenced, standards-based California Standards Test (CST). Students receiving special education services are required to take the same rigorous statewide assessments as all other students including students with severe cognitive disabilities who participate in an alternate assessment as determined by the individualized education program (IEP).  Student participation rates in STAR are a measure for adequate yearly progress (AYP) under the No Child Left Behind Act of 2001 (NCLB). Although the participation rate for students receiving special education services decreased slightly in 2005-06, the overall participation rates are above the SPP AYP target of 95%, with 2005-06 rates of 96.5% for California Standards Tests in English-language arts (ELA) and 96.7% for math.

Special education students’ proficiency in English language arts. While the 2006 STAR Program has four components, only the CST and the California High School Exit Exam (CAHSEE) are used to determine the percent of students scoring at or above the proficiency level for the purpose of meeting AYP. The CST is a criterion-referenced test designed to carefully align with the state’s academic content standards. Grade level specific exams are given each year to all students whose scores are reported in five levels: Advanced, Proficient, Basic, Below Basic and Far Below Basic. Students scoring “proficient” or “advanced” are considered “proficient.” The school as a whole, the district, and all numerically significant subgroups, including the students with disabilities subgroup, at both the school and district levels, are required to show performance at or above the statewide annual measurable objectives in ELA (and mathematics) as the primary factors in meeting AYP.

For grades 2-8, the California Standards Tests in English-language arts and mathematics and the California Alternate Performance Assessment (CAPA) for students with the most significant cognitive disabilities are used to determine the percent proficient.  For grades 9-12, benchmark scores in English language arts and mathematics on the California High School Exit Exam (CAHSEE) and CAPA are also used to determine the percent of students “proficient.” (Performance levels of “advanced,” “proficient,” and “basic” were recently set on the CAHSEE because NCLB requires at least three performance levels for any assessments used to determine AYP.)  These scores, set considerably above the “passing” score for the CAHSEE, were found to be statistically equivalent to the “proficient” score on the CST for grade 10. Thus, the English language arts CST gives quite an accurate picture of progress on AYP literacy measures, as well as state standards, for students with disabilities.

SPP Indicator #3C—Statewide Assessments: Proficiency Rate of Students Receiving Special Education Services.          In response to NCLB, California set Annual Measurable Objectives (AMOs) in ELA and mathematics proficiency. Set by CDE, AMOs are the minimum percentages of students who are required to meet or exceed the proficient level on the state assessments used for AYP. Unfortunately, California did not meet targets for ELA proficiency for students receiving special education services in 2004-05 or 2005-06. Only an average of 17% of students receiving special education services scored proficient or above in ELA on statewide assessments in 2004-05; this rose to 20% in 2005-06, an increase, but still a failure to meet the AMO benchmark of 23% (averaged for all types of districts) as shown by the graph below. 

Averae ELA Proficiency at Sites Receiving TA
545 Sites who received 3 or more days of TA

	2003-04	2004-05	2005-06	2006-07	2007-08	2008-09	2009-10
AMO Benchmarks	~12%	~23%	23%	~23%	~34%	~45%	~56%
Statewide, All Students	37%	42%	45%
Statewide, Students with Disabilities	15%	17%	20%
AMO Benchmarks	~12%	~23%	23%

California AMOs targets for ELA proficiency are level between 2004 and 2007 and then increase yearly to meet the NCLB goal that by 2014 all students will be proficient in ELA. While the improvement shown between 2003-04 and 2005-06 is heartening and shows progress and promise, the challenge to California is clear. In the coming years California must increase the rate of improvement dramatically in ELA proficiency for students with disabilities in order to meet the AMO for AYP (23% in 2006-07, 34% in 2007-08, 45% in 2008-09, 56% in 2009-10, and 67% in 2010-11—year four of the proposed SPDG program). It is equally clear that, to meet AYP in the future, not only do achievement levels for students with disabilities need to improve, especially in the high school grades, but the achievement levels for all students must improve.  Although general education exceeded the AMO benchmark for ELA proficiency in 2005-06—23% target vs. 45% actual—by 2009-10 the target reaches 56% proficient or above. Historical improvement rates portend that all schools and districts have a pressing need to increase ELA proficiency for all students.

The graph above also shows the deep and troubling disparity between the percent of all students that are proficient or above in ELA (45%) vs. SE students (20%) in 2005-06.  Equally concerning is the huge disparity in the percentage of students in SE vs. GE scoring in the lowest possible levels, “Below Basic” and “Far Below Basic.” As the table below depicts, in 2005-06 there were 59%very low achievers in SE compared to 23% in GE for 4th graders; and 70% in SE vs. 28% in GE for 7th graders.  In addition to the disparity in achievement in ELA between GE and SE students, students with disabilities actually lost ground over time, beginning in 4th grade with 20% scoring proficient or above, going to 11% in 7th grade, and ending in the 10th grade with only 6%. The figures are shown for 3 sample grades within each schooling level, but the data holds true across all grade levels from 4th -11th. This is a clear demonstration of need for immediate action in literacy at the middle and high schools, in particular.           

SPP Indicator #3A-Statewide Assessment-percent of districts meeting AYP objectives. Multiple factors determine whether a district makes AYP:  participation in statewide assessments, proficiency in ELA, and proficiency in math. The measurable and rigorous target for 2005-06 is 52% of districts meeting AYP objectives for the disability subgroup. Although California just met this target with 53.9% of districts meeting AYP, this will be more difficult in the future as AMOs for ELA rise. A reasonably high percent of districts are meeting AYP in math, and as discussed above, California is meeting AYP targets for participation in statewide assessments. As shown in the table below, the one factor that is preventing the greatest percentage of districts from meeting AYP for the students with disabilities subgroup is proficiency in ELA.

3.A. Percent of districts meeting Adequate Yearly Progress Objectives (disability subgroup)
		2004-05	2005-06
Participation	ELA	97.5%	78.4%
	Math	95.1%	86.7%
	Both	95.1%	75.6%
Proficiency	ELA	58.4%	66.9%
	Math	83.6%	87.7%
	Both	56.6%	64.4%
Overall	All AYP	53.5%	53.9%

Note:  Participation rates in 2005-06 dropped as students who use modifications are no longer counted as participating.

Moreover, the need to improve ELA proficiency is pressing for all district types. There are slightly different AYP Annual Measurable Objectives in ELA for different types of districts (i.e. unified, county office of education, high school 7-12, elementary, high school 9-12.)  The requirement for meeting Annual Measurable Objectives benchmarks applies to all significant subgroups, including students with disabilities. As shown by the table below, “on a statewide basis, all district types did not meet the 2005-06 targets” (CDE, 2007).

Percent of Students with Disabilities ELA Proficient by District Type
District Type	ELA  Target 2005-06	ELA  Actuals 2005-06
Unified, COE, High Sch. 7-12	23%	19.6%
Elementary	24.4%	20.8%
High School 9-12	22.3%	16.7%

            SPP Indicator #4A—Suspension and Expulsion. After ELA proficiency, the second area where California needs improvement is SPP indicator #4A, suspension and expulsion.  The target is that no more than 10.4% of districts will have an overall suspension or expulsion rate greater than 1 percent (more than 1% of children suspended or expelled for more than 10 days in one school year.)  In 2005-06, 17.9% of districts—or 154 districts—had a SE rate of expulsion or suspension of more than one percent.  Suspension and expulsion are devastating to student achievement. Research shows that the one variable that is known to increase student achievement is “seat time” or undisturbed, productive classroom time (Scott, Barrett, 2004). Intervention that provides training in positive behavioral supports (PBS) for education personnel is a serious need in California to reduce the very real cost to all students of behavior problems in the classroom and to meet SPP targets for suspensions and expulsions.

SPP Indicators #1 and #2-Graduation and Dropout rates, respectively. Interrelated to indictors for academic achievement and suspension/expulsion are SPP Indicators #1-graduation rates and #2- dropout rates. Higher academic achievement and lower suspension/expulsions rates boost graduation rates and lower dropout rates. Statewide in the 2005-06 school year, 61.6% of students receiving special education services graduated with a regular diploma, up 4.8% from the previous year. Although California did meet its SPP target for graduation rates, unacceptably, 38.4% of students with disabilities did not graduate high school. In future years, graduation benchmarks will be more difficult to reach as key performance indicators for SPP targets increase by district type (for high school 9-12 from 50% in 2005-06; to 67% in 2010-2011; to 76% in 2011-12 (SPDG year 5).

As with graduation rates, California met the 2005-06 school year SPP target for dropout rates but the benchmarks will get more difficult to meet in the next several years. Currently, 85% of districts met or exceeded the established annual benchmark for students with IEPs staying in high school. Again, the benchmarks vary by district type, but for high school 9-12 districts the 2005-06 benchmark was 6.8% dropouts and this was easily met with only 3.1% of students reported by these districts as dropped out or moved and not known to be continuing. By 2010-2011, (SPDG year 4) the benchmark for this district type decreases to 2.2%. The good news is that California’s drop out rate for students with disabilities has steadily declined since a level of 7.3% in 1997-98. However, data reveal that at present African-American and Native American students with disabilities are disproportionately dropping out of grades 7-12, which raises issues related to Indicator #5-Least Restrictive Environment and Indicator #9-Disproportionality Overall.

            SPP Indicator #5- Free Appropriate Public Education (FAPE) in the Least Restrictive Environment.             Historically, students with disabilities have not received equal treatment in the education system, leading IDEA 1997 to focus on two specific areas of equality—access to the general education curriculum and relative rates of punishment in terms of suspensions and expulsions. IDEA 2004 and its implementing regulations aligned to NCLB and made many changes to how the states monitor and report educational outcomes. Among other things, it included new provisions relating to the state’s monitoring for disproportionate representation of racial and ethnic groups in special education and the extension of the “highly qualified teacher” requirement to special education instruction, both of which are discussed below.

California needs to continue to increase collaboration between general education (GE) and special education (SE) as a cornerstone of systems change. To increase student achievement it is critical that students with disabilities receive educational services equal to their general education peers.  Access to the core curriculum by educating students with disabilities in the least restrictive environment (LRE) is measured by the percent of children with IEPs ages 6-21 in the following classifications:

• Indicator #5A-removed from the regular classroom <21% of the day;
• Indicator #5B-removed from the regular classroom >60% of the day; and
• Indicator #5C-served in public/private separate schools, residential placements, homebound, or hospitalized.

California’s rigorous and measurable 2005-06 baseline data, targets, and actuals are shown for indicators 5A, 5B, and 5C in the table below.

Indicator	Baseline (Dec. 2004)	Target	Actuals
5A	49.2	At least 51.1	50.4
5B	24.6	Not more than 24	24.2
5C	4.4	Not more than 4.3	4.3

Statewide LRE 5A and 5B targets were not met, although the actual figures are statistically very close.  However, over time it will be more difficult to meet all these targets, as 5A will increase (% removed <21% of the day should increase) and the other two targets will decrease. By SPDG year four (2010-2011) the target 5A indicator will increase from 51.1% to 76%; 5B: from 24% to 9%; and 5C: from 4.3% to 3.8%.  In addition, African-American students are over represented in the 60 percent removal group (5B) and in the served in separate facilities group (5C). California has identified the need to continue devoting increased interventions, attention, and resources on working with students with disabilities in the LRE in order to meet future targets and eliminate over representation of African-American students (CDE, 2005, p. 49).

SPP Indicator #9- Disproportionality Overall. As part of its Quality Assurance Process, California has long used data specific to racial/ethnic disproportionality when monitoring districts. With development of its SPP, CDE continues to monitor all local educational agencies (LEAs) to measure the percent of districts with disproportionate representation of racial/ethnic groups in SE that is a result of inappropriate identification.  Although baseline data and targets have not been approved as yet for this new SPP indicator, based upon prior disparity index measures, African-Americans continue to be most over-represented and Asian-Americans most under-represented—the exact opposite of the make-up of college undergraduates.

SPP Indicator #8-Parent Involvement. California currently collects parent involvement information in a variety of ways: through monitoring processes, including parent input meetings and parent surveys, and through parent support organizations such as Parent Training and Information Centers (PTIs) and Family Empowerment Centers (FECs).  For 2005-06, CDE added a question to the parent surveys to collect data relevant to SPP Indicator #8:  the percent of parents with a child receiving SE services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities.  In 2005-06, 69% of parents reported that schools facilitated parent involvement.  This will serve as baseline data and the initial target. Thereafter, the target will increase from 69% responding favorably to 74% in 2006-07; 78% in 2007-08; 82% in 2008-09; 86% in 2009-10 and 90% in 2010-2011 (SPDG year 4). Obviously, California needs to make real and continuing progress in facilitating parent involvement in order to continue to meet targets.

            Recruiting and Retaining Highly Qualified Special Education Teachers. NCLB and IDEA 2004 mandated that new teachers in core academic areas be highly qualified beginning with year 2002-03, and that all teachers in core academic areas be highly qualified by 2005-06 (NCLB, 2002). In general, “highly qualified” means teachers must hold a bachelors degree, a state credential or internship credential and demonstrate subject matter competence in the core academic subjects taught. Although CDE has undertaken a comprehensive study to address the issue of better aligning highly qualified teacher requirements with related elements of quality teacher preparation, NCLB and IDEA require different subject matter competence for SE teachers than the California credential currently requires. While NCLB/IDEA subject matter competency is tied to the particular classroom assignment of a teacher, California’s Education Specialist Credential authorizes the teaching of all subjects in the special education setting.

In this and other respects, NCLB and the IDEA amendments have increased the certification demands for SE teachers. Of course, the chronic shortage of fully certified SE teachers is well known.  For more than two decades issues related to SE teacher shortages and attrition have concerned policymakers and administrators (Billingsley, 2004). Although the number of all teachers prepared each year is sufficient to meet marketplace demands, teacher turnover, especially during the first five years, makes recruitment a Sisyphean task (Darling-Hammond, 2001). According to the Center on Personnel Studies in Special Education at the University of Florida, the national annual SE teacher attrition rate is 13.5% (Honawar, 2006).

In California, teacher turnover presents a significant problem in terms of placing highly qualified teachers in all classrooms. The Public Policy Institute of California, a private, non profit organization dedicated to independent, nonpartisan research, found that 13% of new teachers left public schools in their first or second year of teaching; 22% left by the end of the fourth year—higher for single subject credential holders at 27%. If these patterns continue, about 25% of new teacher hires will be replacements for other recently hired teachers who have left.  (In its study, leaving is defined as two consecutive years without public school employment, as somewhat different and more conservative measure than many Federal studies) (Public Policy Institute of California, 2006). When new teachers are hired to replace leaving teachers, fewer fully credentialed teachers are in the classroom. This is because newer teachers are less likely to be fully credentialed than more experienced teachers. In 2004-05, 25% of newly hired teachers lacked full credentials. However, among teachers with five years of experience, only 11% lacked full credentials and after ten years of teaching experience that share dropped to 2% (Ibid).

Current and anticipated personnel vacancies and shortages: According to California’s Basic Educational Data System (CBEDS), the current (2005-06) and anticipated (2006-07) personnel vacancies and shortages are roughly the same and estimated at 3,500 special education teachers needed, or ~16% of the total teacher personnel shortage. The Center for the Future of Teaching and Learning estimates that California will be facing a shortfall of ~60,000 teachers by 2008 and ~106,000 by 2014 due to the looming retirement of “baby-boomer” age teachers, new requirements for teacher quality, and increasing teacher attrition (CRTL, 2004).

Number of preservice and inservice programs: To attempt to meet this shortage there are 74 preservice California Alternative Certification Internship Programs, most University-affiliated such as SPDG partner, the California State University, Los Angeles (CSULA) program. In addition to Internship preservice preparation programs, California has 58 fully accredited preservice SE preparation programs at the state’s public and private universities and colleges. 

Teacher retention is addressed through inservice programs, but because of California’s extremely large educational system, it is not possible to report with certainty the exact number of inservice programs. One of the largest CA inservice programs for teacher induction and support is the CA Beginning Teacher Support and Assessment (BTSA) Program. This statewide program is organized into 6 regional clusters and provides on-going facilitated mentoring and support through guided inquiry for California’s beginning teachers. Additionally, California is divided into Special Education Planning Areas (SELPAs) by geographical regions. These planning areas are state funded to provide inservice programs for special education personnel. It is estimated that the SELPAs provide ~2,000 inservice programs per year. School districts also provide inservice programs to all of California’s teachers. Estimating only 1 inservice program per month, across California there are ~116,280 school district inservice programs per year. Early childhood special education programs in California are state funded to provide inservice for their personnel. It is estimated that at least 200 inservice programs for preschool personnel occur each year. In addition to these programs, and without including the current SIG2 work, it is estimated CDE provides over 3,500 inservice programs per year at the state, regional, or local level. 

Recent research points to teacher quality as a very important factor, if not the single most important factor, in determining student performance. Therefore, if the national goal of providing an equitable education to all children is to be met in California, new and additional resources must be dedicated to preparing and retaining high-quality teachers.

a.3. How identified weaknesses will be addressed: See section b immediately below. 

 (b)  Significance 

b.1. Likelihood that the project will result in system change or improvement. As discussed in the corresponding sections below, the proposed California SPDG program is likely to lead to system change or improvement because: 

• All professional development (PD) activities and interventions take a systemic approach; 
• Program activities address identified state needs in accordance with the State Performance Plan and Annual Performance Report;
• The SPDG continues and/or scales up SIG and SIG2 program components already showing success in creating system change or improvement; and
• There is a significant impetus and broad-based support for change. 

b.1.A. SPDG professional development activities/interventions take a systemic approach

For the SPDG program, CDE, with guidance from the ISES stakeholder group, has chosen to scale up or initiate high quality PD activities that take a systemic approach toward improving outcomes for students with disabilities.  All SPDG PD will meet the NCLB definition of PD for high quality teacher preparation and will be high quality, sustained, intensive, classroom-focused, founded on scientifically-based research; and aligned with and directly related to state academic content standards, student academic achievement standards, and assessments.  In addition, all PD is addressed to one or more of the CDE-approved core message areas which convey fundamental scientific-based principles: The California Reading Initiative and Special Education in California, Positive Behavioral Supports for a Safe and Healthy School, Collaboration in Education, Family Partnerships, Transition: School to Adult Life, IDEA 2004, and Least Restrictive Environment (LRE). Core messages are posted on the current SIG2 web along with bibliographies of supporting literature and research at http://www.calstat.org/cores.html 

The systemic nature of the change activities, introduced below, is evident from the following characteristics:

• GE/SE collaboration is a primary goal; GE and SE staff will learn together in all activities;
• Professional development will be delivered to school teams that include GE and SE teachers, as well as administrators, paraprofessionals, parents, counselors/psychologists and program specialists—not individual teachers or types of teachers;
• Interventions will be school-wide or district-wide—not a single curriculum delivered to one age group;
• Leadership Sites will be identified for their success in systemic change over 2 years; and
• Both the State Leadership Institute and the Regional Leadership Institute will focus on how to facilitate change and improvement at a systems level to increase student performance.

One aspect of the SPDG’s systemic approach deserves extra attention—increasing collaboration between GE and SE.  Each of the proposed SPDG professional development activities involves increasing collaboration between GE and SE, sometimes called inclusive collaboration. The scientific or evidence-based literacy and PBS professional development activities, the State Leadership Institute, the Regional Leadership Institutes, the Leadership Sites—will all invite, support, and often require the participation of both GE and SE administrators, teachers, paraprofessionals, and parents/family members. Collaboration between GE and SE produces teachers who are more capable and willing to serve students with disabilities in the GE classroom and helps California meet IDEA mandates for student access to GE curriculum and participation in GE settings (Shippen, Crites, Houchins, Ramsey, & Simon, 2005). It fosters an attitude of accountability for all students and moves the school culture toward a sense that “they are all our kids.”  Almost 30 years of research and experience has demonstrated that the education of students with disabilities can be made more effective by having high expectations for students with disabilities and ensuring their access to the general education curriculum in the regular classroom to the maximum extent possible.

b.1.B. Significance of the activities to address the identified needs.

The chart below is a summary of the identified weaknesses/gaps or state needs and the state’s response. The rationale for selecting strategies for inclusion in the SPDG included: 1) the likelihood they will result in important and quantifiable system change and improvement, 2) their demonstrated success in improving personnel knowledge/skills, 3) their strong foundation in research and effective practice, and 4) their capacity to be “scaled up” within districts and statewide to meet the needs of California’s large and diverse population.

Identified gap/weakness-To improve student achievement and proficiency in ELA, literacy (SPP Indicators #3A, #3C)
	State response and SPDG activity:
		ERIA (Effective Reading Intervention Academy)
		Leadership Community: State Institute, Regional Institutes, Leadership Sites T/TA
		Centrally coordinated core message TA
Identified gap/weakness-To reduce suspensions and expulsions  (SPP Indicator #4A)
	State response and SPDG activity:
		Positive Behavioral Supports or BEST (Building Effective Schools Together) T/TA
		Leadership Community: State Institute, Regional Institutes, Leadership Sites T/TA
		Centrally coordinated core message TA
Identified gap/weakness-Need to increase percent of students with disabilities in Least Restrictive Environment (SPP Indicator #5)
	State response and SPDG activity:
		ERIA and BEST  T/TA
		Leadership Community: State Institute, Regional Institutes, Leadership Sites T/TA
		Centrally coordinated core message TA
Identified gap/weakness-To assist schools in facilitating parent involvement (SPP Indicators #8)
	State response and SPDG activity:
		Family Participation Fund
		Parent Training Centers & CA Association of Family Empowerment Centers outreach
Identified gap/weakness-Need to increase the number of highly qualified SE teachers
	State response and SPDG  activity:
		Cal State Univ. LA education specialist alternative route certification intern program
		TEACH California web-based SE teacher recruitment project

Multiple Effects of Activities:  California’s chosen SPDG activities are proven strategies with multiple potential positive effects. There is overlap where chosen scientific or evidence-based activities will contribute to improving outcomes for students with disabilities in more than one SPP Indicator area. For example, improving student literacy skills will improve student achievement as measured by state assessments in ELA (Indicator #3A & #3C) which will in turn increase graduation rates and reduce drop-out rates (Indicators #1 & #2) (Snow, Burns, & Griffin, 1998; Slavin, Karweit, Wasik, Madden, & Donal, 1994).

Another example is facilitation of parent/family involvement.  Increasing parent/family involvement (Indicator #8) may help parents determine if their child is appropriately identified for SE services, which may in turn decrease the racial/ethnic disproportionality for SE services overall (Indicator #9). As yet another example, improving teacher knowledge and skills in scientific or evidence-based positive behavioral supports (PBS) may result in both 1) children identified for SE services due to behavioral problems staying in the GE classroom a greater percentage of the day (Indicator #5); and 2) reducing suspensions and expulsions (Indicator #4). 

California SPDG significant activities introduced below include:

• Professional development, training and technical assistance (T/TA) in scientific or evidence-based interventions in literacy (ERIA) and positive behavioral supports (the BEST schoolwide discipline program);
• The Leadership Community (State Leadership Institute, Regional Leadership Institutes, and Leadership Site activities) characterized by collaboration between GE and SE;
• Centrally coordinated TA centered around evidence-based  “core message” areas;
• Technological innovations, including databases and tools for decision-making;
• The Family Participation Fund and PTI parent/family outreach and survey;
• The CSULA education specialist credential alternative route intern program; and
• Web-based technology to recruit SE teachers through TEACH California.

Scientific or evidence-based professional development activities in literacy using ERIA:   Piloted in 2004-05 to great success with other CDE funding, the Effective Reading Intervention Academy (ERIA) scientific-based literacy program supports schools in assessing struggling students, identifying their specific literacy skills that need improvement, and delivering effective interventions to assist them. The initial phase of ERIA is based on the work of Dr. Anita Archer and takes a “best practices” approach to training school sites in how to develop an intensive reading intervention program for struggling elementary and middle school students. ERIA training features scientific-based curricula and techniques that work schoolwide, including for students receiving special education services and for English language learners.

For the SPDG, school site teams of teachers, administrators, and program specialists will participate in the training. The training will prepare participants to utilize a Response to Intervention (RtI) model of student assessment, placement in intervention programs, intervention, and monitoring through ongoing assessments. After the initial school site training, an onsite coach will be selected—such as the reading specialist, an administrator or classroom teacher—to keep the momentum going and to ensure model fidelity. The efforts of the onsite coach will be supported by an outside coach who will work regionally with a cohort of sites. With special expertise in ERIA implementation, the outside regional coach will provide on-going TA to the sites as they develop and implement the ERIA program.

            Scientific or evidence-based professional development activities in Positive Behavioral Supports utilizing BEST:  The SPDG focus on improving student outcomes is addressed, in part, through Building Effective Schools Together (BEST), a school-wide program based on positive behavioral supports (PBS) research by the National Center on Positive Behavioral Interventions and Supports (an OSEP-funded research center) at the University of Oregon (Sprague & Golly, 2004). Over the past 12 years, Dr. Jeffery Sprague, a key SPDG consultant, and his colleagues at the Institute on Violence and Destructive Behavior have played a seminal role in the development, evaluation, and dissemination of PBS school discipline practices to improve student achievement. Dr. Robert Horner, a PBS leading researcher, is also a key consultant to the SPDG and will enhance development of the BEST coaching component.

The California BEST T/TA program will help schools develop and implement positive school rules, rule teaching, and positive reinforcement systems. Starting with the first SIG project, California identified the need for increased positive behavioral management training for educators and service providers. The state’s most recent SPP Annual Performance Report (Indicator #4-suspensions and expulsions) confirms there is a continuing need for classroom PBS to create a safe and supportive environment that encourages academic achievement and healthy social development.

With SIG Supplement funding in 2002, California piloted a “training of trainers” model with the BEST PBS program. Expert trainers lead by Dr. Jeffery Sprague developed a cadre of BEST trainers who in turn trained school teams over a 2- to 3-year period to implement and maintain effective schoolwide, classroom and individual student interventions. School teams work to complete initial and ongoing needs assessment, choose interventions (e.g., school rules, reward systems, systematic supervision), and use student- and staff-level data to refine and evaluate their efforts (Todd, Horner, Sugai, & Sprague, 1999). In the SPDG, the BEST model will include sustaining and supporting follow-up coaching as described further on pp. 47-48.

The Leadership Community:  The Leadership Community is a community of practice focusing on CDE-approved evidence-based “core message” areas. The Leadership Community will consist of three different SPDG activities: the Leadership Site Award Program, the Regional Leadership Institutes, and the State Leadership Institute. Each is described below.

The Leadership Site Award Program: This statewide program willidentify successful models of inclusive collaboration and exemplary or model practice in one or more SPDG “core message” areas. Started with SIG and continued in SIG2, each year high performing middle and high school sites will compete for an award through a rigorous, formal written application including school level data, descriptions of promising practice or use of scientific or evidence-based interventions, and their readiness for system change. Participating sites must commit to:

• Ongoing professional development activities;
• Providing 3 presentations at Regional Leadership Institutes or local training events;
• Participating in the State Leadership Institute and online wrap-around services; and
• Developing program Web pages to be posted on the SPDG website (www.CalSTAT.org).

            Site-to-site TA: Leadership Sites will actively participate in the learning community by providing TA to other schools and districts and by receiving TA to continually improve their own program. In the past, Leadership Site TA included site-to-site peer mentoring in the form of site visits, presentations, and online virtual tours of awarded sites. Leadership sites will be financially supported with grant funds to share their work statewide. They will receive a $5,000 subaward, a stipend to help off-set the cost of attending the State Leadership Institute, and six days of centrally coordinated TA to allocate and use in areas of greatest need.

Regional Leadership Institutes:The Regional Leadership Institutes are part of the larger picture of school and district site capacity building. They will assist in developing a responsive service delivery system by bringing together regional teams of administrators, teachers, other professionals, individuals with disabilities, and parents/families to effect system change. It is a process and not merely an event. The Regional Institutes will bring together teams of administrators, GE and SE teachers, specialists, paraprofessionals, and parents/families to work collaboratively to rethink, redesign, and restructure their organizations to educate students with disabilities in the least restrictive environment and improve outcomes. Over time participants will develop collaborative, scientific-based educational approaches and strategies for better serving students with disabilities. The regional learning community will stay connected through face-to-face meetings, online conferencing, conference calls and follow-up TA days. Regional Leadership Institutes will be coordinated by LEAs called “host sites” chosen for their leadership in local capacity building. They will plan a “mini-conference” with follow-up on-line or telephone support in one of the targeted “core message” areas and adhering to the State Leadership Institutes’ annual meeting design, will:

• Adopt a whole systems approach (working as part of a systemic effort to improve student outcomes, including diversity of representation, offering opportunities to work in teams representative of local systems, and focusing on system change);
• Build connections over time (connecting the learning community year round through online conferencing/conference calls, and focusing on full team support);
• Emphasize shared leadership and capacity building (viewing participants as experts, providing parents with an equal voice, structuring to foster connections, using model sites as presenters, using participants to lead round tables); and
• Deliver high priority, school-focused and process-focused “core message” content in a format encouraging integration and application (developing content strands, focusing keynote presentations on models/skills for systems change, allowing teams time to integrate learning).

State Leadership Institute: As it is now, the State Leadership Institute will be an annual face-to-face meeting bringing together existing and new Leadership Sites and Regional Institute host sites, for intensive training in the “core message” areas. The goal of the State Leadership Institute will be to develop collaborative systems involving GE and SE educators and parents/families in implementing effective, scientific-based educational programs and strategies.  The three-day Institute includes training workshops, small group meetings, role-alike cross-site meetings, and usually a nationally recognized author and speaker on system change, such as Dr. Margaret (Meg) Wheatley. Core message collaboration and teambuilding skills will be a cornerstone of Institute training. Similar to the Regional Leadership Institutes, each site will bring a team of administrators, GE and SE teachers, education specialists, counselors, and parents/family members; as studies show this combination to be most effective (Harriott, 2004).

To foster an experience that has greater breadth and depth then a three-day event can provide, participants will be asked to engage in an intense wrap-around online conference the week before, the week of the event, and the week after. During the February 2006 State Institute 240 postings were made in the online conference under six discussion topics. A newsletter, online conferencing, and other web-based programs will help sustain the Leadership learning community as described in the “project design” section, p. 48.

Centrally coordinated TA involves support and centralized coordination of TA services to educators (including institutes of higher education), education administrators, education-related service providers, and parents/families.  The centrally coordinated TA system was developed during the first SIG to meet the need for easy-to-access TA.  All SPDG-supported TA will be linked to one or more CDE-approved “core messages” listed above on pp. 20-21.

SPDG TA may include a broad range of activities—any training, facilitation, modeling, problem-solving, trouble-shooting, or coaching that supports the efforts of IHEs, SELPAs, LEAs, schools, families and parent organizations, and other organizations to improve outcomes for students with disabilities. TA funds are paid to the district/SELPA/Parent Training Institute, etc.—not to the TA provider. This has resulted in significant leveraging of funds and the requesting site may hire their own consultant or have their needs matched by SPDG staff with a consultant from the over 500 in the current SIG2/SPDG consultant pool. TA may be delivered on-site, on-line, by phone, or via fax or hard-copy.  For example, a school may receive literacy training in the use of the DIEBELS assessment and then request TA for teachers in the use of findings for student placement.

During the SIG and SIG2 programs, staff developed a system for delivering TA that will continue for the SPDG.  When requests for TA are received, SPDG staff will respond with a type of triage process. The TA Coordinator will first refer the caller to the general guidelines for TA posted on the website at http://www.calstat.org/TA.html. If the request conforms to the guidelines, s/he will then discuss potential alternative resources to meet the TA need. S/he will award a number of TA days, usually 1-2 but up to 6 maximum. TA requests will be prioritized according to the following criteria, those that:

• Promote collaboration between GE, SE, and parents/families to improve special education;
• Leverage other resources for regional impact and a spread of effect;
• Use scientific or evidence-based practices in the TA delivery, as well as content; and
• Help meet SPDG objectives and help distribute SPDG resources across California.

Activities facilitating parent/family participation and the Family Participation Fund. Individuals with disabilities and parents of persons with disabilities will be employed by SPDG subcontractors, participate on the ISES stakeholder and SPDG planning groups, and will be highly involved in a variety of activities aimed at helping schools and LEAs facilitate parent involvement. First, the project will work to increase the number of parents/family members taking an active leadership role on school site teams at state and regional institutes. By learning together, school personnel and parents will model meaningful parent involvement in school improvement efforts. Parents will also receive the latest information about evidence-based instructional practices and will be part of the planning for implementing those practices for continuous improvement of services for all students.

Second, the state will continue its commitment to meaningful, representative family participation in decision-making bodies at all levels of the education system. Established under the first SIG, the Family Participation Fund (FPF) will provide funds to offset expenses incurred by low-income and under-represented families participating in decision-making and planning bodies at the local, regional, and state levels.  For the SPDG, use of the FPF will be scaled-up through collaboration with the Parent Training and Information Centers which will conduct outreach in geographical areas where the Fund has been under-utilized and to groups under-represented in Fund use. Also, the FPF will work with the California Youth Leadership Forum alumni network to outreach to youth with disabilities trained in systems level collaboration who may want to participate in educational system reform with help from the FPF. 

Increasing the number of highly qualified special education teachers. Through expansion of the very successful CSULA educational specialist credential alternative route intern program, a state-approved program, the SPDG will increase the number of highly qualified SE teachers and will retain those teachers at least three years after completing the program.  SPDG funding will increase program capacity to recruit, monitor, and advise SE credential candidates already possessing a bachelor’s degree. In addition, some SPDG resources will improve CDE’s TEACH California teacher recruitment web site. The functionality of the web site will be enhanced through the creation of an online registration system.

b.1.C. Continuing and/or scaling up successful SIG2 program activities.

The SPDG continues and/or scales up SIG program components already proven successful or showing success in creating system change or improvement in California—a huge accomplishment for a very populous state with limited educational resources. The section below summarizes evaluation results for activities conducted under the most recent grant, SIG2, including those supported through SIG-leveraged funding. These positive results influenced SPDG planners in designing the SPDG program.           

Scaling up effective scientific or evidence-based professional development activities in literacy using ERIA:             With SIG2-leveraged funding, SIG2 staff coordinated ERIA training of 3 cohorts each consisting of 10 primarily middle school sites with ~10 team members per school for a total of 300 participants. All sites are currently receiving follow-up coaching. The SPDG will scale-up or expand ERIA training by adding one cohort of 10 school teams per year. School teams for each cohort will be from within a single district or local region. At the end of five years, 50 additional school sites will be implementing the scientific-based reading interventions of ERIA together with the existing 30 sites for a total of 80 school sites.

Pilot evaluation data shows ERIA T/TA success in increasing student proficiency in English language arts (SPP Indicator #3). Using 2003-04 as the baseline year, the cohort 1 sites implementing ERIA in 2004-05 and continuing in 2005-06, show very positive changes in student ELA proficiency as listed and shown below.

All Students

• 8 of 9 sites (89%) increased the % ELA proficient by 5 or more percentage points.
• The average % ELA proficient rose 9 percentage points, from 48% to 57%.

Students with Disabilities

• 6 of 9 sites (67%) increased the % ELA proficient by 10 or more percentage points
• The average % ELA proficient rose 14 percentage points, from 13% to 27%!
• In 2005-06 ERIA cohort 1 met the annual measurable objective benchmark in ELA!

Averae ELA Proficiency at Sites Receiving TA
545 Sites who received 3 or more days of TA

	2003-04	2004-05	2005-06
ERIA Cohort 1 Average, All Students	48%	55%	57%
Statewide, All Students	37%	42%	45%
ERIA Cohort 1 Average, Students with Disabilities	13%	17%	27%
Statewide, Students with Disabilities	15%	17%	20%
AMO Benchmarks	~12%	~23%	23%

By design, ERIA T/TA was coordinated with SIG2 activities, meaning ERIA sites participated in the SIG2 Leadership Community with its emphasis on increased collaboration between GE and SE. For cohort 1, eight out of nine school sites accessed TA from SIG2 individual Leadership Sites in the core message area of “collaboration” and specifically in the area of collaborative GE and SE literacy programs. In fact, two school sites so fully implemented ERIA that in fall 2006, they applied to the Leadership Site Award Program and became SIG2 Leadership sites! On an evaluation survey, one administrator said of ERIA T/TA: “It has raised teacher awareness of the needs of specific reading skills that their curriculum does not include… We are now working as a team to meet the needs of our students.”   

Scaling up effective scientific or evidence-based professional development activities in positive behavioral supports using BEST schoolwide discipline management program:  As with the evidence-based literacy program, ERIA, the BEST schoolwide PBS evidence-based T/TA program is proving successful in California not only in reducing suspensions and expulsions (SPP Indicator #4) but shows promise in supporting increases in the percent of students scoring proficient or above in ELA on statewide assessments (SPP Indicator #3)!

As of May 2006, the cadre of California BEST trainers concentrated in four regions across California delivered 37 training events to 197 school site teams (cohort 1) involving ~1600 participants, and delivered 139 days of coaching support or follow up TA to cohort 1.  Since May 2006, an additional 51 school site teams (cohort 2) have been trained, with excellent results. In addition, during SIG2, cadre trainers delivered BEST PBS training to over 300 other school site teams in the Los Angeles area outside of SIG2 funding.

The PBS schoolwide training is a primary scale-up activity for the SPDG in several respects. First, a statewide PBS leadership team will be created and meet annually to plan for state unification of PBS intervention strategies, provide oversight of the state BEST initiative, and address issues of sustainability. Second, in addition to sustaining 30 sites from SIG2 cohort 2, regions 1 – 3 will each train five new school site teams per year and region 4 (Los Angeles) will train 15 new teams per year—resulting in a total of 150 new and sustained BEST PBS trained school sites over the grant period. Training for school sites will be concentrated within districts and local regions to promote district-wide implementation. Third, each site will have a site team leader or internal coach as well as an external coach, usually the Cadre trainer who trained them, to provide TA and support in implementing BEST with fidelity.

The SPDG planning team and stakeholder group decided to scale up BEST due to the positive results of the two year pilot program. After two years of cadre training and coaching support, 143 of the 197 sites (73%) have implemented BEST to varying degrees. These include 84 elementary schools, 30 middle schools, 15 high schools, and 14 other schools (charter, continuation, alternative, etc.). Of the 143 sites that have implemented BEST, at least 114 sites (80%) reported having “Partially or Fully Implemented” BEST, meaning they have implemented BEST to a great degree but may still be engaged in the ongoing work of implementation.

Comparing the baseline year (2003-04) and the most recent year of data (2005-06 data) from the partially and fully implementing sites for which data were available, reductions were seen at the majority of sites in Office Discipline Referrals (ODRs) and Suspensions.

• 54% of sites showed a reduction in ODRs with 48% realizing reductions of 20% or more.
• 63% showed reductions in suspensions with 48% realizing reductions of 20% or more.

In addition, school site administrators report increases in ELA proficiency which they attribute to changes in their school and classroom environment due to BEST implementation. The 2005-06 data for 97 BEST/PBS sites (partially and fully implemented) using 2003-04 as the baseline year shows positive change in ELA proficiency at the majority of sites.

All Students

• 91% of sites increased the percentage (%) ELA proficient
• 41% of sites increased the % ELA proficient by 8 or more percentage points (meeting or exceeding the statewide average improvement during that time period).
• The average % proficient for all students rose 6 percentage points, from 36% to 42%.

Students with Disabilities

• 67% of sites increased the % ELA proficient
• 41% of sites increased the % ELA proficient by 5 or more percentage points (meeting or exceeding the statewide average improvement during that time period)
• The average % ELA proficient rose 5 percentage points, from 11% to 16%.

Note: Taken as a group, these BEST/PBS sites began with a lower % of students ELA proficient than the statewide averages. On the subject of systemic change, one BEST school site administrator said: We have almost complete buy in from the staff all the way to the bus drivers! We started to implement the program in common areas so if there was resistance, it wouldn’t seem to single anyone out. It has passively carried over into the classrooms. So far so good!

Scaling up effective Leadership Community T/TA:  The successful Leadership Site Award Program will be scaled up in the SPDG with five new sites awarded each year for a total of 25 new Leadership sites over the five year grant period.  In addition, the 28 current Leadership sites will continue to be supported through SPDG-leveraged funds. 

The sheer amount of T/TA provided in California during the first 2 years of SIG2 through just the Leadership Community and centrally coordinated TA is enormous. Accomplishing the activities shown in the chart below to good results truly requires a well-organized, well-coordinated, and harmonious group such as the in-place California SIG/SPDG team.

TA Events by Activity Area
303 events with 12,870 Participants from 947 teams

	events	participants	teams
Leadership Sites Providing TA	119	2239	42
Regional Insittute Events	41	3298	526
Regional Institute Follow-up	40	2692	243
Centralized TA	103	4641	136

During SIG2 California teachers, administrators, site teams, and LEAs requested TA most often in the core message area of increasing collaboration between GE and SE, followed by literacy and PBS.  This is another good reason to scale up activities in these areas.  

Leadership Community training event evaluation results show that participants reported a 45% increase in knowledge, and 83% of participants reported a high likelihood that they will use what they learned. Overall, T/TA events were highly rated for 2005-06—an average of 4.4 on a 5 point scale. Three-month follow-up evaluation data shows 77% of respondents reported implementing what they learned, 48% implemented the training content repeatedly, 84% shared what they learned with others, and 57% shared it with others repeatedly.

An effect on student achievement using California Standards Test (CST) data for ELA proficiency was examined for sites that received three or more days of TA during SIG2 (545 school and districts), comparing 2003-04 (baseline) to 2005-06. Recognizing that many factors influence student academic achievement at school and district sites beyond TA provided through SIG2, evaluation findings are impressive as shown by the graph below.

Averae ELA Proficiency at Sites Receiving TA
545 Sites who received 3 or more days of TA

	2003-04	2004-05	2005-06
TA Sites Average, All Students	43%	48%	50%
Statewide, All Students	37%	42%	45%
TA Sites Average, Students with Disabilities	17%	20%	23%
Statewide, Students with Disabilities	15%	17%	20%
AMO Benchmarks	~12%	~23%	!23%

Students with Disabilities

• 82% of TA sites increased the % ELA proficient (average increase=6 percentage points);
• TA sites starting above the statewide average in the % ELA proficient increased an average of 4 percentage points, from 27% to 31%;
• TA sites starting below the statewide average in the % ELA proficient increased an average of  9 percentage points, from 7% to 16%; and
• 40% of TA sites met the ELA proficiency annual measurable objective for 2005-06 as compared to 30% in 2004-05.

All Students

• 90% of TA sites increased the % ELA proficient (average increase=7 percentage points).

 

• 95% of sites met annual measurable objective ~23% proficient.

            Family Participation Fund: The SIG2 Family Participation Fund (FPF) was successful in facilitating participation of low-income and under-represented parents and family members of children with disabilities. During SIG2, 3,272 FPF stipends were provided to 466 family members to support their participation in local, and regional, or statewide decision-making activities, events, and groups. Between 2005-07, 83% of FPF stipend recipients were non-Caucasian—53% self-identified as African American. The FPF was also successful at reaching out to low-income families. 93% of stipends went to recipients who reported their household income as less than $30,000 annually—83% reported incomes of less than $10,000 annually.  

b.1.D. Impetus for change:  As change theorists have known for years, the likelihood of systemic improvement is dependent on a complex of factors, not just the viability of the change design.  In addition to the solidity of the project design in terms of system improvement, California is primed for change, as evident in the following factors:

• Leadership: The CDE Director of Special Education, Mary Hudler, is absolutely committed to the kind of systemic change that will improve outcomes for students with disabilities. She promulgates and practices a “shared leadership” vision and has the skills necessary to make that vision a reality. She is a recognized and trusted leader who listens to ISES feedback and adjusts appropriately. This type of leadership is an excellent harbinger for project success.
• Broad-spectrum consensus: The proposed SPDG program has been built on a consensus model, involving key agencies and individuals representing virtually all the concerned constituencies in the state. The ISES’s training in data-based decision making has enabled the very diverse group to reach consensus with relative ease compared to the first SIG years.
• Business as usual is not working:  Parents, educators, administrators, SELPA directors, and others participating in the SIG2 Leadership Community noted several factors that provide an impetus to change. These are:
• standardized testing, annual yearly progress and the desire to stave off referral to program improvement for California schools;
• the high and rising costs associated with special education services combined with the lack of positive results from special education services;
• the lack of service alternatives for students needing increased academic support but not considered disabled or not qualified to receive special education services; and
• the high turnover rate for special education teachers and the poor school environment and lack of support for these teachers.

            In summary, SPDG significant activities were selected based upon prior SIG program findings and are identified as improvement strategies in California’s SPP. They will achieve system change through high quality professional development because they: 1) take a systems approach involving both GE and SE professionals; 2) are scientific or evidence-based; 3) take highly effective strategies to scale, and 4) capitalize on the foundation and infrastructure established in California over the last eight years.

(c) Quality of the project design 

c.1.  Clearly specified, measurable goals, objectives, and outcomes.

Goal:  The central purpose of the FY 2006 State Personnel Development Grant (SPDG) is to continue to reform and improve California’s system of personnel preparation and professional development (PD) to increase student achievement and improve outcomes for students K-12 with disabilities.  To address this complex and multi-faceted purpose, CDE, guided by California’s Improving Special Education Services Committee (ISES or Partnership), formally adopted as the goals for the SPDG the research-based “required elements of an effective educational system now and in the future” listed in IDEA Part D, Subpart I §1451(a)(6). CDE then prioritized the needs, chose evidence-based approaches to address the needs, and assigned to SPDG funding those crucial elements required for systemic change that could not be locally funded.

Outcomes: This process resulted in six SPDG outcomes targeting high quality professional development (PD) under NCLB in the areas of literacy and positive behavioral supports, scaling-up successful SIG2 activities, parent/family participation, and SE teacher recruitment/retention. From the prioritized needs, the activity focal points for systemic improvement were developed as measurable objectives. All the objectives relate directly to outcomes linked to the core messages and reflect OSEP’s GPRA performance measures. Each objective has quantifiable performance measures and benchmarks of success.

Objectives and outcomes: The objectives and outcomes are detailed below by outcome areas. A logic model is included in the Appendix.

Outcome 1: SPDG activities are aligned with SPP improvement strategies

Objective 1: To provide personnel development T/TA that is aligned with improvement strategies in the California State Performance Plan (SPP), as demonstrated by:

• Performance Measure 1a: To deliver SPDG activities that are named as SPP improvement strategies in each of the three SPP Improvement activity areas: 1) monitoring and stakeholder meetings, 2) T/TA, and 3) utilizing data in decision-making.

Outcome 2: The percentage of personnel receiving PD in scientific or evidence-based instructional practices is increasing or scaling up within districts and statewide

Objective 2: To increase the percentage of personnel receiving PD in scientific/evidence-based instructional practices by scaling up successful SIG2 activities, as demonstrated by:

• Performance Measure 2a: Increasing ERIA implementation to 10 school sites each year;
• Performance Measure 2b: Increasing BEST implementation to 120 school sites;
• Performance Measure 2c: Increasing scientific or evidence-based core message T/TA provided by 25 Regional Leadership host sites and 53 Leadership Sites; and
• Performance Measure 2d: At least 80% of PD T/TA will be based on scientific/evidence-based instructional/behavioral and systems change GE/SE collaboration practices.

Outcome 3: Professional development activities are improving the knowledge and skills of personnel in delivering scientific or evidence-based instructional and behavioral practices

Objective 3: To provide PD activities/training based on scientific/evidence-based practices, specifically ERIA (Effective Reading Intervention Academy) and BEST (Building Effective Schools Together), improving personnel knowledge and skills, as demonstrated by:

• Performance Measure 3a: Increasing the knowledge/skills of ERIA and BEST instructional/behavioral practices among training participants by at least 35%;
• Performance Measure 3b: Increasing the knowledge/skills of personnel in delivering instructional/behavioral practices as measured by at least 50% of ERIA and BEST training participants repeatedly implementing what they have learned; and
• Performance Measure 3c: Increasing to 75% the sites implementing ERIA or BEST in which personnel are observed implementing learned practices with fidelity.

Outcome 4: Personnel knowledge and skills in scientific or evidence-based practices are supported and sustained through ongoing comprehensive practices

Objective 4: To implement strategies and practices that sustain the knowledge and skills of personnel in the implementation of scientific/evidence-based practices, as demonstrated by:

• Performance Measure 4a: Personnel from at least 80% of the schools implementing ERIA or BEST will access 3 or more sustaining activities after 2 years in the program;
• Performance Measure 4b: At least 95% of PD training activities in ERIA or BEST will be sustained through ongoing and comprehensive practices;
• Performance Measure 4c: At least 60% of school sites accessing supporting and sustaining activities will implement ERIA or BEST with fidelity within three years;
• Performance Measure 4d: The percentage of all students and students with disabilities who score proficient or advanced on the California Standards Test will increase by at least 5 percentage points in at least 70% of ERIA school sites implementing with fidelity;
• Performance Measure 4e: The percentage of discipline suspensions will decrease by at least 20% in 50% or more of the cadre-trained BEST school sites; and
• Performance Measure 4f: The project will develop and/or disseminate at least 2 tools or instruments for supporting and sustaining improved practice or system change.

Outcome 5: Parents are involved with schools and the broader educational system

Objective 5: To facilitate parent/family involvement with schools and the educational system as a means of improving services/results for children with disabilities, as demonstrated by:

• Performance Measure 5a: 70% of Family Participation Fund stipends will go to parent/families reporting annual household incomes of less than $30,000; and
• Performance Measure 5b:  At least 70% of school site teams attending the State Leadership Institute will include the participation of parents/family members.

Outcome 6: Special Education teacher recruitment and retention is improving

Objective 6: To improve the recruitment/retention of quality SE teachers as demonstrated by:

• Performance Measure 6a: The number of total visitors to the teachcalifornia.org website will increase 10% per year after outreach in year one;
• Performance Measure 6b: The education specialist candidate enrollment in the CSULA Special Education Intern Program will increase by at least 25%; and
• Performance Measure 6c: 90% of CSULA education specialist intern program graduates will be employed in public schools for at least three years after program completion.

c.2.  Relationship of the project design to the identified needs. In section “a,” five primary needs were identified; those needs—and the SPDG project response—are delineated below.

            Need 1: To improve student achievement and proficiency in ELA, literacy (SPP Indicators  #3A, #3C) Response: The SPDG will focus on reading intervention, long recognized as fundamental to academic success for students with disabilities. Through Early Reading First, Reading First, and other federal, state and local initiatives to generate preschool reading readiness and a strong foundation of reading skills in elementary schools, the state is beginning to see improvement in ELA proficiency testing in elementary schools. Efforts for improving reading at the middle and high school levels have been less intensive and have had minimal impact to date on ELA test scores. Students with disabilities show a clear decline in ELA achievement as they enter middle school and continuing through high school. Thus, the SPDG targets high quality literacy T/TA to middle school educators.

Need 2: To reduce suspensions and expulsions (SPP Indicators #4A, #9) Response: Based on documented effectiveness, the evidence-based BEST schoolwide positive behavioral support (PBS) program will continue from SIG2 and be scaled up in the SPDG. BEST professional development training will be supported and institutionalized statewide by the California cadre of ~18 BEST trainers/external coaches and ~150 trained onsite coaches. Cadre trainers and onsite coaches will meet annually to increase their content knowledge/skills and receive updates on data monitoring methods; other sustaining activities are listed on pp. 47-48.

Need 3: To increase percent of students with disabilities in Least Restrictive Environment (SPP Indicator #5) Response:  All proposed SPDG activities together and individually are designed to create an educational environment that institutionalizes free and public education (FAPE) in LRE. In addition, CDE addresses the need to improve in this SPP Indicator area, reflective of the core message supporting LRE, through various programs outside the SPDG, including the WestEd LRE Resources Project.

Need 4:To facilitate parent involvement (SPP Indicators #8 & #9) Response:The primary SPDG outreach and partnership activity with parents/family members is the Family Participation Fund (FPF), which will engage family members, targeting those from diverse backgrounds and supporting their participation within the education system while building their capacity as advocates for their children and other students with disabilities. This is a system-focused goal aimed at increasing the presence of parent/family representatives of individuals with disabilities on key initiative committees at decision-making levels. The FPF will be administered through the California Association of Family Empowerment Centers (CAFEC), which coordinates a statewide network of local Family Empowerment Centers (FECs)s.  FECs work directly with families of children with disabilities to provide education and empowerment.  Also, SPDG “mini-grants” to California’s five PTIs will fund additional outreach and their participation on the ISES Task Force on the Development of Parent Data Collection and Interpretation. This task force will work at adapting the parent survey materials produced by the National Center for Special Education Accountability Monitoring for use in California. 

Need 5: To increase the number of highly qualified SE teachers. Response:  The SPDG responds to the need to recruit/retain highly qualified SE teachers with funding to expand the California State University, Los Angeles (CSULA) intern certification program for SE teachers under the direction of Dr. Andrea Zetlin. Begun in 1998 as one of only 3 programs nationally, the CSULA alternative program has helped alleviate SE teacher shortages (McKibbon & Ray, 1994). In partnership with the Los Angeles Unified with its 768 schools and 83,972 special education students, the CSULA intern program has been particularly successful in recruiting men (avg. 33%), Hispanics (avg. 42%), and African-Americans (avg. 14%) (Honawar, 2006).

With SPDG funds, CSULA will add recruitment personnel to expand SE teacher outreach and recruitment efforts to LAUSD paraprofessionals (teaching assistants/non-credentialed teachers) working in the schools, 2nd career individuals, and persons who are interested in its low incidence certification programs in visual impairment and physical/ health impairments available through distance learning.  To enter the program, CSULA interns must have a BA/BS, pass the CBEST, show subject matter competence, and be employed as a SE teacher. 

The University intern program and employing school district each provide extended guidance/supervision while the interns complete all educational coursework and fieldwork requirements for the SE credential.  The school district will provide the intern with a full time teaching assignment, a site-based mentor, and district support (i.e., staff development, resources, and materials). University faculty will model state-of-the-art teaching and facilitate intern discussions of “real world” teaching problems within the context of theory and practice (Karge, Lasky, McCabe, & Robb, 1995).

Support for improvement will be provided by both University supervisors and district mentors.  The site-based support providers will follow a peer-coaching model for one-on-one support for the interns in teaching, planning, assessments, and consulting with parents.  They will also demonstrate lessons and share resources. CSULA supervisors will observe interns’ instruction, provide feedback, monitor progress, and maintain regular contact with the employing school principal and mentor. Every 10 weeks, interns and their mentors will attend a 3-hour PD workshop conducted by CSULA internship program director, Dr. Andrea Zetlin and her staff. Once every quarter interns will meet with CSULA program staff to monitor progress toward completing credential requirements. The collaborative effort of the University and district ensure that the interns will experience growth in professional competencies. 

A modicum of SPDG resources will be dedicated to improving the TEACH California website, a teacher recruitment strategy administered by CDE. Targeting potential special education, mathematics and science teachers, a password protected registration feature will be added to allow interested individuals to sign-on and receive answers to questions and messages related to credential requirements, teacher preparation programs, financial aid, and jobs. LEAs will be able to contact individual prospective teachers in their geographic area. The prospect for new teacher recruitment is enormous as the website currently receives an average of over 1,200 visitors per day.

c.3.  Activities constituting a coherent, sustained program of training in the field.

The creation and maintenance of literacy, positive behavioral supports (PBS) and leadership learning communities targeting scientific/evidence-based interventions and core message areas produces a coherent, sustained program of PD with wrap-around follow up TA and support. 

For the ERIA school-wide literacy program, the SPDG will support the pilot program’s 3 cohorts, sustain program implementation for new/old sites, and ensure model fidelity through:

• on-site coaching by a school site team member;
• outside coaching by a regional coordinating coach working with each site within a cohort;
• an annual one-day “booster” training in program content for all site teams;
• an annual two-day statewide workshop for onsite team coaches to increase their skills;
• TA to support data collection and other evaluation activities;
• Four 2-hour teleconference trainings; and
• Comprehensive online wrap-around conferencing.

After initial team training, the BEST program sites will be sustained and supported through the same activities as listed above for the ERIA sites. The California cadre of BEST trainers/external coaches will provide follow-up TA and assist each site as they implement their BEST programs—an evidence-based design that provides the intensive support required for success. A newly created state PBS leadership team will provide planning and direction for PBS statewide. Both ERIA and BEST school sites/LEAs will also receive TA through the SPDG learning community, as well as through the online activities described on the next page. 

The SPDG-funded Leadership Community creates a community of practice that generates and sustains system improvement. Through the State Leadership Institute, Regional Leadership Institutes, and Leadership Site Award program, school site teams involved in collaborative change efforts come together with other educators and parents/family leaders to share learning, develop skills, and plan for action. Participants stay connected over time through the use of a variety of communication and information-sharing strategies. The State Leadership Institute alone involves up to four weeks of virtual conferences that take place before, during, and after the face-to-face meetings. Additionally, an ongoing, interactive website features participant-driven conferencing that focuses on priority topics and virtual site visits to awarded programs and Institute events. The learning community has grown to ~700 participants with ~300 participants attending the State Leadership Institute held each year in a different area of the state.

Sustaining Support through Online Activities and Other Communications. Sustaining support for the literacy, positive behavioral supports, and leadership learning communities is critical. Sites involved in ERIA, BEST, Leadership, and State/Regional Leadership Institutes are all supported with wrap-around follow up TA and cross-site connections including:

• Leadership Community online conferences with specific discussion topics;
• ERIA and BEST online conference room for each cohort and for the entire community;
• Teleconferences and expert interview podcasts;
• Community Network database of site profiles and leadership community events;
• Software providing data tools for tracking student and site/LEA outcomes;
• RISE Library (with SPDG leveraged funding);
• Special EDge newsletter (with SPDG leveraged funding); and
• The project website at www.calstat.org which meets 508 accessibility standards, is coded on WebAIM standards, and is aligned with General Services Administration guidelines.

c.4.  Project design reflecting up‑to‑date knowledge from research and effective practice. As noted above, the SPDG effort will focus on specific, scientific/evidence-based core messages and interventions related to the objectives, as detailed in the discussion below.

Outcome 1: SPDG activities are aligned with SPP improvement strategies

California’s SPP was a primary organizing factor for the SPDG.  The SPP already includes most, if not all, SPDG scientific/evidence-based interventions as activities to improve outcomes in 3 areas: 1) monitoring and stakeholder involvement—this is the ISES group and its SPDG evaluation task force; 2) T/TA—this represents the majority of SPDG evidence-based activities as described below; and 3) data-driven decision-making.

Both the TED Dissemination TA and California Standards Test Charting Programprovides schools and LEAs with data-driven decision-making tools; these activities will be supported and maintained by the SPDG. In the initial SIG and SIG Supplement, the evaluator developed the Training Evaluation Database (TED) for the project and then adapted it to meet local SELPA/district needs. TED is a user-friendly one-stop database to track participation, create e-mail lists, and print event rosters and nametags for T/TA events, as well as producing evaluation reports for specified data ranges and levels of aggregation with a “click.”

The California Standards Test (CST) Charting Program developed during SIG2 allows schools/districts to quickly chart their ELA test scores from the CDE website compared to statewide ELA results. State level ELA scores are charted by SIG2 staff. At present, schools and districts are encouraged to generate multi-year charts to measure progress in ELA.

Outcome 2: The percentage of personnel receiving PD in scientific or evidence-based instructional practices is increasing or scaling up within districts and statewide

CDE and the ISES believe that in-service training is essential to increasing the quality of personnel and better informing parents/family members. 90% of the SPDG budget is devoted to this effort which includes the following activities: ERIA, BEST, the State Leadership Institute, the Regional Leadership Institutes, the Leadership Site Award Program, and the comprehensive core message TA through centrally coordinated TA, Site-to-Site TA, the Family Participation Fund, data collection/analysis tools, and other technology-based TA. SPDG professional development meets the NCLB definition of high quality teacher preparation, and in-service TA adheres to evidence-based strategies for maximizing effect in adult learning.

In-service PD is the traditional vehicle for increasing personnel skills and knowledge (Boyer, 2003). Both the scope and quality of PD experiences appear to be associated with SE teacher quality. Teachers whose recent in-service experiences covered a broad range of specified topics have higher aggregate teacher quality scores than those whose recent PD covered fewer topics. Likewise, teachers who perceived their recent district-supported PD as relevant and of high quality had higher teacher quality scores (Carlson, et al., 2002).

The need for in-service training is especially important for SE teachers, as their attrition rates exceed those for GE teachers by about one-third, and more continuing teachers transfer out of special education than in (Boe, et al., 1997). In 1998, one third of entering SE teachers lacked standard certification for their assignments, as did 10% of all SE teachers (Boe, et al., 1998).

Decades of research have clarified the structure for highly effective PD. Hocutt notes that “Several instructional methods appear to result in improvements in academic outcomes for students with mild disabilities. The more promising involve on-going teacher training, teacher planning time, and administrative support.” (Hocutt, 1996).

Outcome 3: Professional development activities are improving the knowledge and skills of personnel in delivering scientific or evidence-based instructional and behavioral practices

Scientific or evidence-based ERIA literacy program: ERIA features scientific and evidence-based curricula and techniques to assess all students, identify specific skill areas of need (oral reading fluency, decoding, etc.) and deliver interventions and combinations of interventions to meet those needs. Literacy assessments provide reliable data to identify struggling readers regardless of their SE status—an intentional rejection of the “waiting to fail” model. In addition to standardized test scores in ELA, assessment tools focus on identifying specific literacy skills of decoding and fluency that impact proficiency. The San Diego Quick Assessment measures student word recognition or decoding skills (Lintz, 1975; Sargent, 2002; Smith & Harrison, 1983); and other assessments—Read Naturally (Hasbrouck, et. al., 1999) or Reading Fluency Monitor (Ihnot, 1991)—measure fluency.

ERIA trains personnel in delivering a range of scientific or evidence-based reading intervention programs, which are then purchased and implemented by sites to meet the specific literacy skill needs of their students as revealed by the assessments. Each program includes pre-post/placement tests and structured curricula. Decoding interventions include: REWARDS which improves multisyllabic word reading for intermediate and secondary students (Archer et. al, 2004; Vachon & Gleason 2004; Shippen, et. al., 2005) and Phonics for Reading for lower level readers, the elements of which match the National Panel of Reading’s (Langenberg, et. al., 2000) research-based findings. Fluency interventions include: Read Naturally, which focuses on teacher modeling, repeated reading, and progress monitoring and has been validated through numerous control group studies, many involving special education students (Hasbrouck, et. al., 1999); and the Six-Minute Solution which employs two research-based and validated techniques—repeated reading and peer assisted learning (Therrien, et. al., 2006).

The ERIA literacy program is implemented in stages, usually over a period of years. A graphic illustration of ERIA Stages of Program Development and Implementation is included in the Appendix. Onsite and outside coaches help sustain implementation with fidelity at the sites as explained above in section c.3., p. 47.

Scientific or evidence-based Positive Behavioral Supports: Many school practices contribute to the development of antisocial behavior and violence, including:

• Ineffective instruction that results in academic failure;
• Failure to individualize instruction to adapt to individual differences;
• Inconsistent/punitive schoolwide or classroom behavior management practices;
• Unclear rules and expectations regarding appropriate behavior;
• Failure to correct rule violations and reward adherence to them;
• Lack of opportunity to learn/practice pro-social interpersonal and self-management skills;
• Failure to assist students from at-risk backgrounds to bond with the schooling process (Sprague, et al., 2001).

These factors are all amenable to change using positive and preventive methods (Mayer, 1995; Sugai & Horner, 1994; Walker, et al., 1996). Unfortunately, school personnel have a long history of applying simple and unproven solutions to complex behavior problems (e.g., office discipline referrals, suspensions or expulsions). They express understandable disappointment when these attempts do not work as expected but often do not adopt alternative practices (Walker, et al., 1996). The tendency to try to remove the problem student via suspension or expulsion, rather than focus on the administrative, teaching and management practices that either contribute to, or reduce them, is common in our schools (Tobin, Sugai, & Martin, 2000).

Educators in today’s schools and classrooms must be supported to adopt and sustain effective, cost-efficient practices (Sugai & Horner, 1994; Walker, et al., 1996). Effective approaches to preventing the onset and development of antisocial behavior, include 1) systematic and sustained social skills instruction; 2) academic and curricular restructuring; 3) positive, behaviorally based interventions; 4) early screening and identification of antisocial behavior patterns; and 5) school-wide rule teaching and recognition systems (Sprague, Sugai, & Walker, 1998; Sugai & Horner, 1994; Walker, et al., 1996). BEST Behavior and similar or identical intervention approaches have been extensively studied (Embry & Flannery, 1994; Knoff & Batsche, 1995; Taylor-Green, et al., 1997). The effects of the BEST Behavior program are documented in a series of studies implemented by researchers at the University of Oregon (Metzler et al., 2001; Sprague, Walker, Golly, et al., 2002; Taylor-Greene, et al., 1997) and have shown reductions in office discipline referrals of up to 50% per annum, with continued improvement over a three year period (Irvin, et al., in press). In addition, school staff report greater satisfaction with their work. Comparison schools typically show increases or no change in office referrals, along with a general frustration with the school discipline program.

In studies of the techniques and components of the BEST Behavior program, reductions in antisocial behavior (Sprague, et al., 2002), vandalism (Mayer, 1995), aggression (Grossman, et al., 1997; Lewis, et al., 1998), later delinquency (Kellam, et al., 1998; O’Donnell, et al., 1995), and alcohol, tobacco and other drug use (Biglan, et al., 2003; O’Donnell, et al., 1995) have been documented. Positive changes in protective factors such as academic achievement (Kellam, et al., 1998) and school engagement (O’Donnell, et al., 1995) are also proven.

The original Oregon study asked treatment and comparison schools to report frequency of office discipline referrals pre/post BEST.  All treatment schools reported reductions in referrals in the intervention year compared to the baseline and showed greater improvement than comparison schools.  Middle school baseline year referrals ranged from 550-3167 and in the intervention year from 260-2608, making the average percent change a 36% decrease (range = -18 to -53%) compared to 82% increase in the comparison schools (range = -39 - +203%).  Of treatment elementary schools with two years data, the average percent change was a 51% decrease with comparison schools showing a 7.5% decrease (Sprague, et al., 1999).

Outcome 4: Personnel knowledge and skills in scientific or evidence-based practices are supported and sustained through ongoing comprehensive practices

             The principles of effective teacher training, all of which are addressed in SPDG efforts, include programs linked to school-wide efforts; teachers participating as helpers to each other and as planners, with administrators, of follow-up activities; emphasis on self-instruction, with differentiated training opportunities; teachers in active roles, choosing goals and activities for themselves; emphasis on demonstration, supervised trials, and feedback; concrete, ongoing training over time; and ongoing support available on request (Sparks, Loucks-Horsely, 1989).

The SPDG training program devotes a significant share of training time to peer-to-peer instruction in learning community settings. Using highly qualified personnel who have “been there, done that” in successful programs is effective in offering feedback to the community, answering questions, helping plan activities and set goals, discussing problems, and providing on-going support for these research-based activities (Smollar, 1998). Effective training programs also give participants the opportunity to reinforce what they have learned by testing, repeating, and practicing, an element built in to all the SPDG TA activities (Kennedy, 2003).

Clearly, effective training programs must provide follow-up support to participants as they implement newly learned skills. Specific plans for providing follow-up support must be woven into the training, not tacked on as an afterthought. Effective training assumes that the support provided to assure implementation is the second, but equally important, component of the training activity (NTAC, 2000). This support is inherent in all SPDG TA activities, and these various support approaches have been demonstrated as effective, including on-site visits, mentoring or coaching, video review, live video interactions, product review and feedback, and observations; these can be carried out face-to-face or online (NTAC, 2000).

The SPDG addresses the need for ongoing TA through a variety of programs with demonstrated success, including the State and Regional Leadership Institutes, the ERIA and BEST programs, site-to-site TA, and centrally coordinated TA. In addition, resources and information for teachers will be provided through the Special EDge newsletter and the RiSE Library funded through the state as described in section c.3.above, on p. 48. The bottom line is “that special education can be no better – and no worse – than the quality of instruction provided by teachers” (Heward, 2003), and structured TA can improve that quality.

In addition, the “community of practice” model involving collaboration between GE and SE used throughout SPDG activities reflects effective practices in improving personnel knowledge and skills. “If there is anything that the research community agrees on, it is this: The right kind of continuous, structured teacher collaboration improves the quality of teaching and pays big dividends in student learning and professional morale in virtually any setting. Our experience with schools across the nation bears this out unequivocally” (Schmoker, 2004).

 

Outcome 5: Parents are involved with schools and the broader educational system

Decades of research have proven that “when parents become involved, children do better in school, and they go to better schools” (Henderson, Berla, 1994).  This is especially true for students with disabilities, where a national longitudinal study found that “parental involvement in a student's education was powerfully related to student performance. Controlling for other factors, students whose parents were more involved in their education missed fewer days of school and were much less likely to fail courses than were students whose parents were less educationally involved” (SRI International, 1997).

However, “Teachers must be trained to promote effective parent/family involvement in children’s education… It cannot be assumed that teachers will naturally know how to [do this]” (Carter, 2002). And likewise, parents/family members need information and support to work with the educational establishment. “It cannot be assumed that parents instinctively know how to involve themselves in their child’s education. In fact, many parents feel inadequate in teaching roles…and in dealing with schools” (Simmons, Stevenson, & Strand, 1993). This trend begins in the elementary grades and increases as students enter high school, even though “an adolescent’s success is influenced by his or her family even through the last year in school” (Trusty, 1999).

Thus, one of the core messages to be delivered through TA is that educators have ways to work effectively with families and that families must have the skills/understanding to work effectively with educators. Effective working relationships require systematic two-way communication—school to home and home to school (Epstein, 2001). The SPDG will provide this through targeted T/TA.

Another core message in relation to parent/family member involvement is that the system must prepare families to exercise leadership and advocacy in school decision-making. Including parents in assessment, placement, policy-making, and advocacy is recommended to restore the balance of power in parent-professional discourse (Harry, 1992). This is exactly what the SIG Parent Participation Fund does for low-income families.

Outcome 6: Special Education teacher recruitment and retention is improving

Alternatives for teacher certification have helped alleviate teacher shortages in special education, math, and science teachers who are highly qualified (Honawar, 2006). According to Michael McKibbin, the alternative certification officer for the state Commission on Teacher Credentialing, in 2006 nearly 3,100 candidates were enrolled in special education alternative-route programs—a jump of almost 600 from the prior year (Honawar, 2006). Alternative route intern programs are proving to recruit a diversity of SE teacher candidates and produce quality teachers. “On many variables related to teacher quality, special education teachers who earned their certification through alternative routes did not differ from those who earned their certification through more traditional programs” (OSEP, 2000).

Teacher turnover is a significant factor in causing SE personnel shortages. If new teacher retention patterns do not change, then about one-fourth of the roughly 20,000 new public school teachers hired in California each year are replacing recently hired teachers who have left public school employment (Reed, Rueben, & Barbour, 2006). Supporting new teachers with mentor coaches improves teacher retention. “Beginning special education teachers who had mentors that they rated as effective were more likely to remain in special education” (Boyer & Gillespie, 2000). For teachers not fully certified, sustained practical, ongoing support from experienced colleagues is a necessity.  Without these supports, new teachers are ill equipped to manage the daily demands of teaching (Zetlin & Kimm, 2003). The CSULA intern program is using evidence-based mentor support to increase SE teacher retention. The attrition rate of teachers who have gone through the program is ~5% over the past 4 years, while the national annual attrition rate for SE teachers is 13.5%. Expanding the capacity of the CSULA intern program from ~175 interns to 225 interns with SPDG funding is a clear recruitment/retention win.

c.5. Linkages with other appropriate agencies/organizations.

CDE currently has formal MOUs on the delivery of services for students with disabilities with 5 state agencies: Commission on Teacher Credentialing, Department of Developmental Services, Department of Health Services, Department of Social Services, and Department of Rehabilitation. Major linkages with these and other agencies/organizations are within the ISES stakeholder group Partnership, including parents/parent organizations, institutions of higher education, early intervention programs, community- or faith-based non-profit organizations, the Interagency Coordinating Council, etc. Others involved in the delivery of SPDG PD or achievement of SPDG objectives are the California Association of Family Empowerment Centers, the Parent Information and Training Centers, and LEAs (NCOE, LAUSD, Leadership Community members). A table of ISES members is included in section a.1, p. 4.  MOUs and letters of cooperation from organizations linked to SPDG activities are in the Appendix.

c.6. Project as part of a comprehensive effort to improve teaching/learning and support rigorous academic standards for students.

ISES provides a sounding board for resource allocation to support the various improvement activities. The funding behind this comprehensive effort is a patchwork of leveraged SPDG funding through the CDE CalSTAT program discussed in section e.3. The following activities reflect other parts of the plan:

• Increasing participation of students with disabilities in statewide assessments: California Alternate Performance Assessment (NCLB).
• Improving early literacy for all students: Reading First and ERIA (EDD/CDE).
• Improving early intervention/preschool SE services: Supporting Early Education Delivery Systems and SE Early Childhood Administrators Project (IDEA); Early Childhood Content and Performance Standards Project (CDE); Desired Results-Access preschool development (CDE).
• Increasing access to GE: LRE Initiative Project (IDEA).
• Increasing parent/family participation and knowledge/skills: Procedural Safeguards and Referral Services; Parent Training/Information Centers; Family Empowerment Centers (IDEA).
• Improving accountability, data-based decision-making, and data dissemination:NCLB Adequate Yearly Progress (NCLB); Quality Assurance Process; CA Special Education Management Information System (CASEMIS); WorkAbility data collection; CA School Information Services; DataQuest; CA Teachers Integrated Data Educational System (CalTIDES); CA Longitudinal Pupil Achievement Data System (CalPADS).
• Increasing the number/effectiveness of partnerships: IEP Task Force; Qualify Assurance Process (QAP) (IDEA); Autism Advisory Committee (CDE).
• Improving service coordination: a variety of cross-cutting teams within CDE and among agencies and technical assistance centers (i.e. Western Regional Resource Center, National Post-School Outcomes Center, Personnel Center at the National Association of State Directors of Special Education, the National Comprehensive Center for Teacher Quality, CA Comprehensive Center implementing NCLB and Program Improvement (IDEA, NCLB, EDD, CDE).
• Increasing the supply of HQ special education teachers: Strategic Plan for Recruiting, Preparing, and Retaining Highly Qualified Special Education Personnel (funding – IDEA).
• Align CA Education Specialist credential with NCLB HQ teacher requirements (funding—CDE and Commission on Teacher Credentialing [CTC]).

(d) Quality of project personnel 

d.1. Encouragement of applications from underrepresented groups. The California Department of Education (CDE) supports high-quality educational programs for California's diverse student population through its educational leadership role. The Department places great importance on cultivating a diverse staff that understands and promotes intellectual, ethical, cultural, emotional, and physical growth for every student. It is CDE policy, in alignment with Equal Opportunity Employment regulations, to encourage employment applications from underrepresented groups. In the case of California’s SPDG, the only employment involving federal funds will be through Napa County Office of Education (NCOE).

NCOE is subject to all laws governing affirmative action and equal opportunity including Executive Order 11246 and Title IX of the Education Amendments Act. NCOE has an outstanding history of non-discriminatory employment practices for members of traditionally underrepresented groups such as racial/ethnic minorities, women, and persons with disabilities, as documented by annual reports filed with the state; the personnel policies explicitly guarantee “equal treatment and encourage applications for employment from members of underrepresented groups.” Reports over the last decade show full compliance with the affirmative action/equal opportunity plans filed with the state. All staff positions in this proposal will be filled under stringent enforcement of these policies. If vacancies occur during the project, employment of new personnel will be fully subject to these policies.

For the SPDG, NCOE will send electronic job notices to all Historically Black Colleges and federally listed minority institutions, in addition to other colleges/universities and associations of education professionals from underrepresented groups.  All notices will state “Persons belonging to underrepresented groups and minorities are encouraged to apply.”

d.2. Qualifications of key personnel from CDE and Napa County Office of Education (NCOE).

SPDG Administrators:  SPDG administrators are from the highest level of educational leadership and programming in California. All SPDG administrators are currently serving as a Director or Division Director for CDE or the California Department of Developmental Services.

CDE Chief Deputy Superintendent of Public Instruction, Gavin Payne. Gavin Payne, four-year CDE Chief Deputy Superintendent, reports directly to the State Superintendent of Public Instruction. He manages all policy and operational activities, ranging from facilities to instructional materials. He oversees the apportionment of funds to local school districts, and he guides partnerships and relations between CDE and schools, districts, education advocacy groups, the U.S. Department of Education, and others. Mr. Payne came to CDE from staff leadership positions at the California Legislature. He is involved in teacher/administrator professional development through his work on the Board of Directors of WestEd, the Council of Chief State Schools Officers, and Elementary and Secondary Education Act Reauthorization Task Force, and other organizations.   

CDE Special Education Division Director, Mary Hudler.   Ms. Hudler served as the Administrator for Policy and Program Services and Interim Director of CDE’s Special Education Division before being named Director in 2005. In her current position, she presents the public position of the state in relation to educating individuals with disabilities, as well as striving to influence public policy.  Her approach is to involve the entire disabilities community in planning for systemic change to improve outcomes for children and families.

California Department of Developmental Services, Richard Ingraham, M.S.  Mr. Ingraham is the current Manager of the Children and Family Services Branch of the California Department of Developmental Services (CDDS). In this position, he has direct responsibility for California’s Early Start program and Part C of IDEA, as well as other programs and initiatives to support early intervention services. Previously Mr. Ingraham served as the Department’s Chief of Health and Wellness and Community Program Specialist. His background includes service as clinical director, program director, and director of specialist services at various therapeutic facilities and service as a school psychologist. He is also an adjunct professor at the University of the Pacific and holds a Marriage and Family Therapist professional license.

State Designated SPDG Director:Coordinating the SPDG and working directly with the operational sub-contractor is Janet Canning, M.S., an expert Consultant to the CDE, Special Education Division. Ms. Canning provided enlightened and dedicated leadership for California in this role for the first SIG, SIG Supplement, and SIG2. She is a primary organizer and facilitator of the California ISES stakeholder group and its PCSE predecessor. Additionally Ms. Canning is the Special Education Division’s lead staff on NCLB, CDE liaison to the California Commission on Teacher Credentialing, and Coordinator of the Comprehensive System of Personnel Development. She established the Division’s website and trained the current webmaster. Previously, she was Assistant Director for the Oklahoma Department of Education. 

d.3.  Qualifications of project consultants or subcontractors

            d.3.A. Qualifications of organizational subcontractors.  As it has since the first SIG, California will subcontract with the current SIG managers and staff for administrative assistance and implementation of the SPDG.  SIG2 Project Co-Manager, Dr. Anne Davin and her staff are currently employees of the California Institute of Human Services at Sonoma State University, CDE’s SIG2 primary subcontractor. When SPDG is funded, Dr. Davin and her staff will transfer to NCOE where the CalSTAT program is already in place and where the CIHS Director, Tony Apolloni, Ph.D., serves as the Assistant Superintendent. Dr. Apolloni is a driving force behind innovation and improvement in special education services, with a Ph.D. in special education.  He has directed system change projects which have trained over 20,000 state staff, teachers, parents, and service providers in effective educational reform. In this way, California plans to scale-up, expand, and deepen its SIG2 activities under the same leadership that originated the PCSE stakeholder group, collaborated with CDE to design the first SIG program, and managed California’s SIG, SIG Supplement, and SIG2 program. 

Established in 1852 and currently operating with an annual budget of $19.5 million, NCOE has its own significant track record in grant program operation, ranging from early childhood special education to 11-state professional development projects. A few are:

• School-to-Work Interagency Partnership Project (SWITP).  Funded by the U.S. Department of Education, Office of Special Education and Rehabilitative Services, this statewide project improved school to work transitioning for students with disabilities.  NCOE facilitated the central planning committee which resulted in a Memorandum of Agreement approved by the directors of 12 California state departments and the development of a reformed service model.
• Projects REACH I, II & III were training and technical assistance (T/TA) elementary and secondary teacher professional development programs on substance abuse, prevention, and intervention issues. Funded by the U.S. Department of Education, these projects provided training, distance learning, and resources to over 1,000 educators and other school personnel.
• Project TEACH.  Funded by the U.S. Department of Health and Human Services, this project provided training throughout Northern California for a wide spectrum of service providers on the relationship of substance abuse to child abuse and the effects of both on very high risk children, many of whom have disabilities.  Health/mental health, social service, legal, justice, general and special education professionals attended the workshops as community teams.

In addition to the sampling of prior projects above, NCOE is currently a Beginning Teacher Support and Assessment (BTSA) Induction Program contractor providing professional development, training, individual supports, and formative assessment for new teachers, including special education teachers, to improve retention and meet NCLB mandates. In BTSA, special and general education teachers explore principles of practice and reflect on content standards, strategies for working with English learners and special populations, and behavior management.

NCOE also has considerable experience working with school site teams.  Since 2002 NCOE has contributed to systems change as a CDE-approved provider of School Assistance and Intervention Teams (SAIT) services to CDE-designated "low-performing schools." Through SAIT, NCOE has gained experience in working with site/district planning teams in under-performing schools. NCOE uses data to assess the degree of current plan implementation, helps school revise plans, and recommends ways to maximize resources in achieving district goals.

The SPDG key NCOE personnel include the Project Manager and Evaluator.

            Project Manager Anne Davin, Ph.D. is the current SIG2 Project Co-Manager; Co-Manager of California’s Special Education Technical Assistance and Training project, CalSTAT; and Manager of the IDEA T/TA project. Previously, she served as the Division Director of the Taos Pueblo Health Services, Program Director for University of California (UC) Santa Barbara professional psychology training programs, and Program Director for the Human Relations Center. She also served as legislative liaison to New Mexico State Senator Carlos R. Cisneros. A well-published author, her expertise has been particularly helpful in addressing the need for improved positive behavioral management training. Her degree is in psychology.

Internal Evaluator for the SPDG  is Cheryl “Li” Walter, Ph.D. Dr. Walter served as the internal evaluator through Sonoma State University on California’s first SIG and SIG Supplement, and she currently is the full-time evaluator for SIG2. As the Director of Evaluation and Research for CIHS/Sonoma State since 1997, Dr. Walter served as internal evaluator and supervised staff on several education, health, and child welfare grant-funded projects and as an external evaluator under contract to County Offices of Education, and an OSEP national TA provider. She served as Project Director for the Center on Social Services Research and the Bay Area Social Services Consortium at the UC Berkeley, and as the Consultant to the Medi-Cal Managed Care Project for Alameda County. She is an insightful and careful academician. Her 2000 doctorate is from the UC, Berkeley.

The Person Loading Chart below shows time in days per main SPDG activity area by state administrators and key contractor personnel.

	State Administrator	CDE Director	NCOE Manager	NCOE Evaluator
Literacy/ERIA T/TA	8	25	35	55
Positive behavioral supports/BEST T/TA	8	25	35	55
Leadership Community	5	25	85	50
Supporting/sustaining T/TA	5	15	30	45
Parent Involvement	4	10	25	10
SE teacher recruitment & retention	5	15	20	15

d.3.B. Qualifications of consultants.  As mentioned above, a variety of consultants will contract to support the activities, especially training and technical assistance (T/TA) in the targeted core message areas.  For the first SIG, staff developed a significant consultant pool from the nationwide Hilton/Early Head Start Training Project, the Region IX Head Start Quality Improvement Center for Disability Services, and the statewide IDEA Alignment Training Project. Over the years other consultants with expertise in the seven core messages were added. Those with excellent ratings have been retained so the SIG consultant pool now includes 50+ key consultants providing T/TA to schools/districts across the state. Brief bios of just a few of these consultants are included below; vitae are in the Appendix.

Kevin Feldman, Ed.D., is the Director of Reading and Early Intervention for the Sonoma County Office of Education. His major contributions are in building vocabulary and academic English, assisting struggling readers, linking assessment to instruction, developing school-wide reading support models, and differentiating instruction. He is a contributing author for the Association for Supervision and Curriculum Development, the Council on Exceptional Children, Prentice–Hall Language Arts Program, and Scholastic’s Read 180 Intervention Program. Dr. Feldman has taught for 19 years in Special Education at the UC, Riverside, and at Sonoma State.

Jan Hasbrouck, Ph.D. is a literacy coach working with the Washington Office of the Superintendent of Public Instruction and the University of Oregon Behavioral Research and Teaching Group. A former researcher at the Center for Academic & Reading Skills, University of Texas, Houston Health Science Center, Dr. Hasbrouck is a leading ERIA coach. Her 1994 doctorate is in Educational Psychology (emphasis: special education) from Texas A & M.. 

            Robert Horner, Ph.D. Dr. Horner is professor of special education at the University of Oregon and director of Educational Community Supports, a research unit focusing on practices resulting in positive, scientifically-validated change in the lives of persons with disabilities and their families. For 25 years, Dr. Horner has led systems change efforts related to positive behavioral supports, including the BEST Behavior system. He co-directs the OSEP-funded National Center on Positive Behavioral Interventions and Supports at the University.

Sharon Keating, M.A. is an experienced consultant in organizational change, and is a coach, trainer and facilitator who has successfully orchestrated organizational change efforts, designed leadership development programs, and lead work groups. She is a SIG2 consultant in the design and facilitation of Regional Leadership Institutes and the State Leadership Institute. 

Jeff Sprague, Ph.D., is currently the Director of the University of Oregon’s Substance Abuse Prevention Program, the Co-Director of the University’s Institute on Violence and Destructive Behavior, and a Senior Research Associate for the University’s Specialized Training Program. He is an Editorial Board Member for both the American Journal of Mental Retardation and the Journal of Applied Behavior Analysis. His research activities include applied behavior analysis, positive behavior supports, functional behavioral assessment, school safety, and juvenile delinquency prevention. He recently received a 5-year research grant from the National Institute on Drug Abuse to evaluate the effects of a positive behavior support intervention.

Andrea Zetlin, Ed.D. is a Professor in the Division of Special Education at California State University Los Angeles. She joined the faculty in fall, 1989 in the Mild/Moderate Program. She is also the Director of the C. Lamar Mayer Learning Center and the Coordinator of the Multicultural/Multilingual Special Education Masters Program. She is currently Project Director of the CSULA Special Education intern program.

d.3.C. Qualifications of ISES members. California’s Improving Special Education Services stakeholder group (formerly the PCSE) will serve as the advisory body for the SPDG. Grant funds with support the work of the ISES and the primary contractor, NCOE, will staff it. Members of ISES include many parents of children with disabilities. The chart in section a.1, p. 4 lists the membership in terms of required and optional partners by agency affiliation or individual status, and shows the years of their attendance at annual strategic planning meetings.  In addition, most members participate in at least one ISES workgroup on issues such as post-secondary transition, data-driven decision making, etc.

(e) Adequacy of resources 

The SPDG partnership is requesting OSEP funding that will be matched generously by contributions from CDE and the project partners.  These combined funds form an adequate pool to support project activities. 

e.1.  Adequacy of support, including facilities, equipment, supplies, other. CDE will provide as an in-kind contribution the facilities, equipment, supplies, and other resources required for the in-kind staff working on the project.  These staff include:

• Special Education Division Director, Mary Hudler, .1 FTE, as a Project Administrator;
• Deputy Superintendent, Gavin Payne, .05 FTE, as a Project Administrator;
• Special Education Division Consultant, Janet Canning, .5 FTE, as the Project Director;

            Each staff person will be supported with an office, office furniture, telephone, computer, printer or access to printers, and access to copy machines, scanners, fax machines, internet, etc.  Personnel will receive normal office supplies to support their activities, as well as access to the full range of Department services, e.g. printing/publications, contracting, legal services, etc.

The California Department of Developmental Disabilities is providing an in-kind .03 FTE grant administrator, Dr. Rick Ingraham, and will support this contributed staff person with space, equipment, and supplies.

The primary contractor, Napa County Office of Education, will provide similar support to its in-kind and leveraged grant-funded staff, including:

• Project Administration, Assistant Superintendent, Dr. Tony Apolloni, .05 FTE;
• Evaluation assistant, 1.0 FTE;
• Publications and materials director, .6 FTE;
• Technology and distance learning coordinator, .5 FTE;
• Other support staff: 1 project coordinator and 1.32 administrative and fiscal assistants.

            The SPDG NCOE staff will be housed in a modern office building where the SIG2 managers and staff are currently located. They will be co-located with other NCOE sponsored-program staff, including most-importantly the staff of the CDE CalSTAT project, CDE’s SE system improvement T/TA project. The building has individual offices, meeting rooms, distance learning and technologies facilities, large-quantity materials reproduction equipment, and community staff facilities. Both space and other resources are ample to support this project.

Grant funding is requested for normal project-related office supplies and activity-specific project supplies. Only computers and needed peripherals and software for evaluation activities are charged to the grant. All office furniture and other equipment, including telephones, additional printers, scanners, fax machines, copying machines, etc., will be provided as an in-kind contribution by NCOE or CDE.

e.2.  Relevance/commitment of each partner to project implementation and success. Each partner has demonstrated commitment to the ISES and the success of the SPDG through a signed Memorandum of Understanding or agreement located in the Appendix.

The MOU with the required partners, the state agencies responsible for administering Part C of IDEA, etc.indicates that all partners will contribute personnel and will…

• Serve on the California Improving Special Education Services (ISES) stakeholder planning committee and appoint representative(s) to appropriate ISES committees and task forces as may be established;
• Work with ISES members to provide oversight for the California State Performance Plan and its improvement activities that are related to or effect SPDG activities, review SPDG progress annually, and contribute to making data-based recommendations for SPDG priorities and strategic approaches for future years;
• Inform staff, colleagues, and families of the work of the ISES and SPDG, as appropriate;
• Participate in SPDG professional development T/TA activities, as desired/appropriate.

            The primary contractor and LEA, Napa County Office of Education, additionally agrees to work with the CDE to serve as the administrative and operational agency for the SPDG.

The institution of higher education required partner, California State University at Los Angeles, agrees to improve or expand the capacity of CSULA’s Special Education Intern Program to increase the number of highly qualified SE teachers.

A partnership agreement is also in place between CDE and California’s five Parent Training and Information Centers to conduct outreach to a diversity of parents of children with disabilities in their PTI service area, including networking with other parent organizations to prepare and assist parents in accessing the SPDG Family Participation Fund.

Letters of cooperation located in the Appendix from a representative sample of the ISES stakeholder group relate specifically to their role, but generally provide that each partner will: 

• Serve on the Improving Special Education Services stakeholder group (Partnership) and appoint representative(s) to appropriate Committees which may be established;
• Work with other Partnership members to provide oversight for the Plan and Grant, reviewing progress annually and making data-based recommendations for priorities and strategic approaches for the following year;
• Participate in training and technical assistance and work to increase TA to the field;
• Participate in ongoing SPDG evaluation activities, including event, interview and analysis, and follow-up surveys as requested.

            Beyond the significant contribution of in-kind staff, an additional major commitment from CDE is foregoing indirect cost recovery. Because the state’s authorized rate is 22.5% of direct costs, this is a considerable contribution and one that speaks to the acceptance of the SPDG program as a key element in the effort to improve outcomes for students with disabilities and as an integral, essential mechanism for systems change. 

All of the above commitments are clearly relevant to the success of the SPDG and to the goal of improving results for individuals with disabilities.  They also represent a massive commitment of funds to help achieve this goal.

e.3.  Adequacy of the budget to support the project. CDE, NCOE, and the SPDG planning team have had extensive experience in planning and executing large scale T/TA and reform programs, including the SIG, SIG supplement, SIG2, and California’s CalSTAT program, described below. This experience is key in developing a budget that will adequately support the project activities, as it virtually eliminates extensive costs needed to bring staff up to speed. 

As with the earlier SIG programs, SPDG dollars will be leveraged through the CDE-funded CalSTAT program to produce sufficient funding to meet California’s large and diverse needs. CDE will virtually match the SPDG budget with CalSTAT program resources which support 1) the Special EDge newsletter with a circulation of 50,000; 2) the Resources in Special Education (RiSE) library which loans thousands of free documents and materials; 3) on-going support of ERIA literacy school site teams through coaching, booster training, and an annual statewide ERIA coaches meeting; and 4) on-going support of the Leadership sites. This is a sizeable contribution toward the adequacy of the SPDG budget.

Also essential to fiscal adequacy are the literally millions of dollars of in-kind resources and services provided by the Department and the partners. Thousands of hours of pro-bono services are provided by ISES members, augmenting the in-kind contributions enumerated above from the contractual partners and sub-contractor. Without these contributions, the budget would certainly be inadequate to support the scope of activities proposed. The partners’ major contributions, therefore, highlight two important Partnership attitudes:

• first, that the project activities are important, will lead to systemic change, are “doable,” and reflect the missions of the partner organizations; and
• second, that the budget will support the activities and result in meeting the objectives.

The Partnership’s historical experience with the SIG has proven these assumptions true.

e.4.  Reasonable costs in relation to project objectives, design, and potential significance. What is “reasonable” to spend to improve outcomes for individuals with disabilities is probably not an issue upon which Americans would find easy consensus. However, there are indicators of cost reasonableness in California’s approach that demonstrate the appropriateness of the federal investment in relation to return:

• Funds leveraged by the federal investment:  There are several ways in which California’s SIG leverages the federal investment: first, in the significant contributions of the partners, as discussed above; second, with the use of additional state (CalSTAT, Cal TEACH, and teacher intern) funds; and third, in the Family Participation Fund which will provide partial payment of family expenses for participants in policy-making bodies at the local, regional, or state level.
• Cost effectiveness of approach:  Cost-effectiveness features have been built into the activities in several ways:  first, multiple options appear in many activities so that the most cost-effective can be chosen at any point (e.g. technical assistance via phone, fax, or on-site visit; wrap-around follow up TA via Internet); second, some activities are designed to produce data to assist in leveraging future funds (e.g. study of BEST effectiveness); and third, the delivery systems for the Regional Institutes depend heavily on contributions of time by the school host sites. 
• Return on the investment:  This project requests $2,500,000 in federal funds per year for 5 years, prompting a logical question about the return on investment. A recent study by Teacher's College at Columbia University calculated the average cost to the government per high school student who drops out at $127,000 in aid, justice expenses, and lost tax revenue. According to these findings, if just one objective, such as increasing literacy for middle and high school students, results in as few as 20 students graduating instead of dropping out, taxpayer savings will exceed the annual budget of this project. Further, for 2005-06 CDE reports a 23% pass rate for students with disabilities on the ELA portion of the High School Exit Exam. California students must pass the exam to receive a diploma. The GAO has calculated the social cost in aid and lost wages per SE student who drops out of school at $635,000. If only 20 of the ~638,000 SE students in California are given the ability to graduate under this grant, its entire five-year budget will be recovered and then some.
• Institutionalization of reform:  California’s SPDG is not a series of band-aids stretched over existing gaps in service; it is true systemic reform. The evidence-based strategies were chosen from those proven to be effective, proven to be capable of “scaling up,” and proven to generate systems change. The final results will be institutionalized changes in the system that will continue for years to improve outcomes for individuals with disabilities. One example is SPDG support of the BEST positive behavioral support system. The entire focus of this SPDG activity is to ensure that the program is fully implemented and successfully institutionalized at the trained school sites. Again, while some other states may be “ahead” of California in this area, the SPDG will allow the state to make major progress—and institutionalize that progress.

e.5.  Continued support of the project after Federal funding ends.  As discussed in section 1.a., California is facing a great many challenges in education, including declining levels of educational attainment, increasing numbers of English language learners, and an old and dilapidated schools infrastructure. Still, CDE’s intent to continue its funding of high quality T/TA professional development, literacy development and positive behavioral supports programming is clear and bolstered by precedent, in that:

• California has leveraged prior State Improvement Grant (SIG) funding through the CalSTAT program for over 6 years; and
• Many other complimentary programs and research projects are supported and influenced by SPDG activities, such as those set out as part of California’s comprehensive plan for educational systems improvement and change on pp. 58- 59.

In addition, participating LEAs and school sites have committed to significant post-project support by institutionalizing grant-funded activities, such as the middle and high school reading intervention and BEST program, where school sites provide the personnel for training and coaching, the purchase of materials, and the support for instructors, aides, etc. who implement the training. 

The state’s education budget is set from year to year, therefore, it is difficult to claim future support exists for the SPDG project elements.  However, possibilities for future funding of systemic change and improvement of the special education system in California exist through:

• Discretionary Federal IDEA funds may be leveraged to further SPDG activities;
• Title IV, Safe and Drug Free Schools funding under NCLB may fund future PBS work; and
• With SPDG success, state funding could continue through specific legislation to spread the effect.

(f) Quality of the management plan 

f.1.  Management plan to achieve the project objectives on time and within budget.

The SPDG project management structure is based on the experience of project management for the initial SIG, SIG Supplement, and SIG2. The structure incorporates the intent of the ISES stakeholder group to reflect and model interagency collaboration in the project management and operations, as well as in the activities. This intent to model interagency collaboration has been well captured in the management structure in the following ways:

• The management administration will be accomplished by an interagency collaborative team;
• No required contractual agency or organization will receive grant funds for administration, directorship, or coordination; project staffing is through the Napa County Office of Education (NCOE), with a smaller subcontract to the California State University Los Angeles; all other agencies and organizations provide significant staff time and other contributions in-kind; and
• The SPDG partners in program implementation will be able to remain an organization of equal members rather than creating a separate grant management bureaucracy or hierarchy with an executive body and officers.

The proposed SPDG partnership with contracted program management is a very efficient and functional management plan that has met California’s unique needs for the past 8 years.

Project administration: The project will be administered by Director-level personnel from the Department of Education (CDE—both SE and GE divisions, representing Part B) and the Department of Developmental Services (CDDS—representing Part C).  As noted below, the role of the administrators is to attend the bi-annual ISES stakeholder meeting, meet with the CDE Designated Project Director and key contractor staff on a quarterly basis, and examine annual outcomes to ascertain the extent to which the agency mission is being met by the project and to determine ways in which the common missions can be fulfilled through re-direction of resources.  In short, the administration provides annual oversight for the project as a whole but at least quarterly oversight for their internal agency responsibilities. Administrators will also review data provided by the project prior to the ISES’s SPDG Evaluation Task Force meetings in March and October; this review will be followed by a conference call between the administrators to discuss any concerns and/or endorse any changes in direction recommended.

The rationale for this approach is to provide overall, visible project leadership from the division/department level for the Parts B and C lead agencies, while allowing their committed time to be focused on “enlightened self-interest,” that is the operation of the project—and the benefits accrued—in relation to their own department/division. This level of management communicates the importance of the project as an agent of systemic change in California.

Project Directorship:  The Project will be directed by Janet Canning, a California Department of Education, SE Division consultant.  As described again below, she will devote .5 FTE to the direction of the project, including the following duties: providing daily oversight of the primary subcontractors, NCOE and CSULA, attending the ISES partners’ annual strategic planning meeting, attending the two annual meetings of the ISES SPDG Evaluation Task Force, and examining annual outcomes to ascertain ways from the agency perspective to improve services through re-direction of resources and re-focusing of grant funds. She is specifically responsible to ensure that program activities are spread equitably across California and conducted in a timely and comprehensive manner to generate the projected outcomes.

Project Management: NCOE will employ a 1.0 FTE SPDG Project Manager and her staff. This same staff has managed and implemented the SIG1, SIG Supplement, SIG2 and CalSTAT, the CDE SE Division’s state-funded TA program, and this fact ensures very close coordination of services and maximum utilization of resources. The specific job duties of the Project Manager are to:

• conduct daily project oversight,
• supervise staff,
• monitor SPDG subcontractors,
• assure the completion of all required narrative and financial reports,
• assure the successful completion of all SPDG activities and outcomes, and
• represent the SPDG at national, state, and local agency meetings.

The Project Manager will report directly to NCOE Assistant Superintendent Dr. Tony Apolloni. Dr. Apolloni meets bi-monthly with the Project Manager to ensure that the project is on target in terms of budget and deliverables, that interim data shows reasonable progress toward meeting the objectives, and that staff has the resources and support necessary to continue to produce quality goods and services.           

Project Timeline: This timeline is designed to provide a summary of required action steps with milestones in each activity area for year one; it is not a daily management plan. For year 1, it reflects a projected start up of February 2008. Years 2 -5 will have a similar pattern without the start-up activities. 

Activity/Action Step	Q1	Q2	Q3	Q4
Program Administration; ISES stakeholder group
Review and revise as needed all protocols and procedures for SPDG	x

 

With CDE and partners, determine ISES meeting dates in spring and fall	x
Update ISES database, finalize annual invite list, send out date cards		x		x
Develop meeting agenda; mail out meeting packets; register participants		x	x	x
Plan and monitor all meeting logistics; prepare all materials		x	x	x
Milestone: Attend and assist in evaluation of ISES bi-annual meetings	x		x
Participate on ISES work groups, ISES SPDG evaluation task force	x	x	x	x
California ERIA literacy T/TA
Contract with consultants to provide school site training and support	x
Establish communications/re-contract w/ existing ERIA sites	x
Deliver 1-day booster training to site teams and new cohort of teams	x	x	x	x
Milestone: Select/establish one new ERIA implementation site	x
Deliver 2-day ERIA curriculum and implementation training		x
Monitor provision of onsite and outside ERIA T/TA to all sites	x	x	x	x
Deliver two-day statewide ERIA workshop for all onsite team coaches			x
Deliver teleconference trainings and online wrap around conferencing	x	x	x	x
Analyze evaluation data and develop recommendations for improvement	x	x	x	x
California BEST positive behavioral supports T/TA
Contract with IVDB at Univ. of Oregon to provided BEST T/TA	x
Establish communications/re-contract w/ existing BEST sites	x
Deliver 1-day booster training to all 4 existing BEST sites		x
Milestone: Select and establish 30 new BEST school site teams through the existing BEST-LEA implementation sites	x	x	x	x
Cadre Trainers deliver BEST training to 30 new school site teams	x	x	x	x
Deliver two-day statewide BEST workshop for onsite team coaches			x
Deliver teleconference trainings and online wrap around conferencing	x	x	x	x
Analyze evaluation data and develop recommendations for improvement	x	x	x	x
State Leadership Institute
Revise facilitation team information packets; contract with facilitators	x
Revise, design, and plan Institute framework and outline agendas	x
Coordinate outreach and registration		x
Develop and edit meeting materials and packets; post online		x
Develop, facilitate, monitor virtual conferences & other online supports		x	x	x
Milestone: Conduct the 3-day conference, collecting all evaluation data			x
Analyze evaluation data and develop recommendations for improvement				x
Regional Leadership Institutes
Update Regional Leadership Institute application, competition procedures	x
Train readers, conduct competition, make final selection	x
Milestone:  Contract with RLI-LEAs and establish events calendar	x	x
Monitor provision of Regional Institute event wrap-around TA	x	x	x	x
Maintain web site with RLI information for sites to share	x	x	x	x
Ensure full participation in Leadership Community	x	x	x	x
Analyze evaluation data and develop recommendations for improvement	x	x	x	x
Leadership Site Award program
Update leadership site application, scoring rubric and competition rules	x
Train readers, and conduct application reading	x
Milestone: select final sites and secure agreements		x
Work with each site to develop web-based, interactive materials		x	x	x
Develop TA plans for each site focused on core message areas, etc.		x	x	x
Ensure full participation of all sites in Leadership Community		x	x	x
Monitor model site work in development/delivery of Regional institutes		x	x	x
Analyze evaluation data and develop recommendations for improvement	x	x	x	x
Centrally coordinated TA
Milestone: Update/expand existing 500+ member expert consultant pool	x	x
Publicize widely the availability of SPDG TA and how to access	x	x	x	x
Process and track TA requests according to SPDG “triage” guidelines	x	x	x	x
Analyze evaluation data and develop recommendations for improvement	x	x	x	x
Family Participation Fund
Milestone: Contract with CAFEC & 5 PTIs for outreach services	x
Monitor outreach/use for geographic, ethnic/racial, socio-economic reach	x	x	x	x
Conduct assessments, including sample interviews of effect	x	x	x	x
Monitor allocation of stipends to SE and GE committee participation	x	x	x	x
Analyze evaluation data and develop recommendations for improvement	x	x	x	x
Special Education Teacher Recruitment and Retention
CDE contract with CSULA for expansion of SE intern program	x
Plan, coordinate SPDG and CSULA program evaluation plans	x
Monitor CSULA progress in recruiting/supporting SE teacher candidates	x	x	x	x
Analyze evaluation data and develop recommendations for improvement	x	x	x	x
Data-driven decision-making tools: TED & CST Charting Program
Maintain TED & CST programs on central TA and monitor progress	x	x	x	x
Do demonstrations at conferences, Institutes; Provide on-going T/TA	x	x	x	x
Analyze evaluation data and develop recommendations for improvement	x	x	x	x
Conduct SPDG evaluation
With CDE/OSEP finalize evaluation plan	x
Milestone: Identify, develop, field test evaluation tools and measures	x	x
Construct all necessary databases, train staff in data collection, input	x	x
Lead the ISES SPDG evaluation task force, attend ISES meetings	x	x	x	x
Conduct program evaluation, analyze data against performance measures		x	x	x
Participate in OSEP SPDG evaluator teleconferences, planning groups	x	x	x	x
Meet/confer with key program personnel, consultants to review progress	x	x	x	x
Prepare evaluation reports; meet with CDE, partners to review progress		x	x	x

f.2.  Ensuring a diversity of perspectives are brought to bear in project operations. A diversity of perspectives permeates the project at many levels from the school site teams which include parents, administrators, GE and SE teachers, specialists and paraprofessionals; to the Administrative Team with CDE general/special education directors and the Department of Developmental Services Director; to the 300+ participants in the State Leadership Institute. Program T/TA is delivered not by one or two consultants with a particular point of view but by a multitude of professionals and practitioners in the field with a diversity of points of view. The project is overseen by the joint broad-based and very diverse SPP/SPDG ISES stakeholder group; see membership roster in section a.1, p. 4.

(g) Quality of the project evaluation

g.1.  Thorough, feasible, appropriate methods to evaluate goals, outcomes, and objectives.

The following plan details the SPDG evaluation management structure and methods that will produce quantitative and qualitative data, provide performance feedback and periodic assessment of progress toward achieving the intended outcomes, and ultimately enable examination of the effectiveness of project implementation strategies. California’s SPDG Logic Model can be found in the Appendix.

Evaluation management structure: The existing network of evaluation personnel, resources, and the Evaluation Task Force, which has conducted and guided the evaluation for SIG and SIG2, will continue in the SPDG.

(1) SPDG Evaluation Task Force:To ensure the ongoing involvement of stakeholders in monitoring progress toward the objectives and outcomes, an Evaluation Task Force meets for a full day twice a year. Comprised of a broad range of ISES stakeholders, this 15-member group provides guidance and feedback on instruments and evaluation methods, reviews formative and summative data and progress reports, and makes recommendations based on the findings, which are presented to the CDE and ISES twice annually. (A Task Force roster is in the Appendix.)

Improving Special Education Services (ISES). ISES is a broadly representative stakeholder group of approximately 100 participants which meets twice annually to monitor progress and guide the implementation of both the SPP and the SPDG. SPDG progress reports, along with Evaluation Task Force recommendations, are presented to ISES for consideration and further refinement or inclusion as recommendations to CDE related to the implementation of the SPDG. All SPDG Evaluation Task Force members are also members of ISES.  

            SPDG participants and the public. SPDG evaluation reports will also be shared widely with personnel participating in SPDG activities, at annual State Institutes, and ERIA and BEST trainings and are posted on the SPDG website.

SPDG lead evaluator.The SPDG evaluation effort is being directed by Cheryl "Li" Walter, Ph.D., who has been the SIG Evaluator for over seven years. Her role is to gather information and draft evaluation plans; design program evaluations for the activities of the grant subcontractors; develop evaluation instruments and methods; draft progress and summative reports; facilitate meetings of the SPDG Evaluation Task Force; present to ISES; develop data tools; and teach stakeholders to use these tools in the process of making data-informed decisions. She is an active participant in the OSEP SPDG Evaluator’s Planning Workgroup.

The SPDG Evaluator will work closely with Janet Canning, CDE Designated SPDG Director and SPDG Project Manager Anne Davin, Ph.D. who is responsible for day-to-day program operations and implementation. Two full-time internal SPDG evaluation staff will manage the gathering and entry of data and generate SPDG activity reports; their work is supervised by the SPDG Evaluator. This structure will effectively address the SPDG intention to build the capacity of the system to generate and use data to inform decision-making processes.

Evaluation Methods:  The evaluation will provide thorough, appropriate formative and summative assessments, the data reports from which will be shared widely with SPDG partners and other stakeholders.

Formative evaluation: The formative evaluation will target the following continuous improvement issues: What is the number, nature, and quality of project activities actually implemented to date? What aspects need improvement and/or change? What problems are anticipated in implementing the next phase, and how will these be overcome? 

Summative evaluation. The summative evaluation will use the performance measures to establish the degree to which the objectives and outcomes have been met, including the quality of process and products, the degree of program implementation with fidelity, and the impact of SPDG activities on the intended audiences. Summative evaluations will be conducted annually.

The table below outlines the activities and the instruments, data sources, and methods that will be used to evaluate the performance measures as they are detailed in section c.1, pp. 40-43. The performance measures were constructed to address each of Guskey’s five levels of professional development evaluation (Gusky, 2000). Following the table is a narrative description of the methods of evaluation being employed for each of the SPDG activities, including the key evaluation questions and focus, instruments and data sources, data collection timelines, and how data will be shared with program implementers.    

Approval for Use with Human Subjects. Most of the proposed evaluation activities have been approved for use with human subjects; new activities will receive approval prior to being implemented. Reapplication will take place as necessary to maintain up-to-date approvals.     

SPDG Activities	Evaluation Instruments/Data Sources/Methods	Performance Measures
1) All PD T/TA including: ERIA, BEST, Regional Institutes and Follow-up TA, and Leadership Site TA	Training Evaluation Database (TED) Event Core Message Topic Tracking Team and Individual Sign-in Sheets “How Was The Training?” event evaluation survey Online Follow-up evaluation survey	1a, 2abcd, 3ab 2abcd, 4b 2abc, 4a 3a 3b
2) ERIA and BEST	Team Implementation Checklists (TIC) System-wide Evaluation Tools (SET) www.pbssurvey.org and www.pbseval.org CA Standards Test Scores (CST) in ELA CST Charting Program Site Academic Measures Metafile (SAMM) ERIA Student Data Tracking Tool Behavioral Incidence Data Reporting Form (BID) School-wide Information System (SWIS) Online Evaluation of Coaches	2ab, 3c, 4c 2ab, 3c, 4c 2ab, 3c, 4c 4d 1a, 4d 1a, 4d 4d 4e 4e 4a
3) Leadership Community	Knowledge synthesizing documents Site Academic Measures Metafile (SAMM) Site Data Profiles	4f 1a 1a
4) Parent/Family Involvement	Family Participation Fund (FPF) Invoices FPF After-event Evaluations State Leadership  Institute Team Registrations ISES Task Force Roster and Agenda Parent-School Involvement Facilitation Survey	5a 5a 5b 1a 1a
5) SE Teacher Recruitment, Preparation, and Retention	Web Use Tracking Software Online Registered Users Database CSULA enrollment figures CSULA graduate employment retention follow-up	6a 6a 6b 6c

            1) Evaluation Methods for: All Training and Technical Assistance eventsincluding those provided as part of ERIA, BEST, Regional Institutes, Leadership Site TA, and Centralized TA. The primary evaluation questions include: Who is receiving training and technical assistance? Was the training and TA identified as being of high quality and useful by participants? Do participants report having increased their knowledge/skill in the content of the training? Do participants report having implemented what they learned? These questions will be answered using the following instruments, data sources, and methods:

1.1 Training Evaluation Database (TED). Over the course of SIG and SIG2, California has developed a Filemaker Pro-based Training Evaluation Database (TED). Information on each training and technical assistance (T/TA) event provided is entered into TED, which enables tracking of the Core Message scientific- or evidence-based instructional and behavioral practices targeted in the training (in most cases, events must focus on these practices to meet Core Message criteria for SPDG funding approval), as well as who is providing and attending the trainings. For each event, individual and team sign-in sheet information, including names, roles, schools/districts, and email addresses, is entered into TED. TED also has event evaluation entry, follow-up survey, and automated reporting capabilities.

1.2 End-of-Event Evaluation: How Was the Training? Anonymous end-of-event evaluations are collected from participants. (A sample of this tool is located in the Appendix.) The response rate is 77%; completed participant evaluations are required to be submitted along with presenter invoices to receive payment. Using a 5-point Likert scale from “Low” to “High,” participants are asked to rate their level of knowledge/skill in the training focus area prior to the event and after, assign an overall rating to the event, and provide qualitative feedback on how the training could be improved. The rating difference between prior and current knowledge/skill divided by prior generates the percent change, e.g., a 1 point increase, from a rating of 3 to 4 represents a 33% increase. Once entered into TED, the database is capable of generating automated reports by event, trainer, core message area, and time period. (A sample TED Event Summary is in the Appendix.) Over 50,000 training and TA participants have completed How Was the Training? thus establishing expected patterns of reported knowledge/skill gains (1.0 point) and overall event ratings (4.5 rating). Throughout the year, program staff monitors the quality ratings and written comments for events and specific trainers by comparing the targeted data to these norms. Reports on each event are emailed to event organizers and presenters, and the data is used by the Evaluation Task Force to generate recommendations for continuous improvement.

1.3 Follow-up Evaluations. Follow-up evaluation surveys are conducted in the form of emails with links to a brief web survey; these are sent to participants approximately 3 months after each training to determine whether they report having implemented what they learned at the training. Participants who do not respond receive up to two reminder emails. The response rate on valid email addresses is approximately 35%. Items are rated using a 5 point Likert scale from “Not At All” to “Many Times.” Qualitative data on what worked well and why, and what barriers they might have encountered implementing their learning in practice are also elicited. (A sample follow-up survey is in the Appendix.)  Follow-up evaluations have proved particularly effective in providing an understanding of barriers to implementation and enabling adjustment of the content of future trainings to address those barriers. Completed web survey data are automatically uploaded into TED, which has the capacity to generate automated reports (responses are reported only in the aggregate), that are then distributed and used in the same ways as the end-of-event evaluations in the continuous feedback and improvement cycle.

            1.4 Dissemination of TED. During SIG2, TED was re-programmed and distributed for adoption by school districts, County Offices of Education, SELPAs, and other SPDG partners, enabling immediate use of evaluation data for data-informed decision-making at the local level. During the SPDG, TED will continue to be updated to add functional capabilities and report capacity requested by local level users. TED installation and training will be available to approximately 7 new sites per year. Follow-up training in how to use the advanced data reporting features will be provided to all new and continuing TED users upon request. 

2. Evaluation Methods for: ERIA and BEST.  While the content focuses of these two major SPDG activities are different (literacy and positive behavioral supports), the manner in which they are being delivered and evaluated is quite similar. In addition to the training and TA event evaluations of quality, initial knowledge/skill, and implementation described above, the primary evaluation questions for ERIA and BEST include: Are the sustaining activities offered being accessed by personnel and school sites? Are school sites and personnel implementing the key elements of instructional and behavioral practices with fidelity? Are the school sites which are implementing with fidelity seeing positive changes in student outcomes? Are localized concentrations of sites in districts/counties scaling up implementation of these practices? These questions will be answered using the following methods:

            2.1 Review of records. Tracking of the utilization of sustaining activities will occur through examining records from three sources: TED, Coach’s reports, and web tracking statistics. These sources include data on the participation of personnel and site teams in the coaching, training for site team leaders, booster trainings, Leadership Site TA, data tools, teleconferences, and the online community activities available to sites on an ongoing basis following completion of their initial ERIA or BEST training. For the purposes of the performance measure, “accessing supporting or sustaining activities” means participating in three or more activities by the end of their second year in the program, and annually thereafter.

2.2 Team Implementation Checklist (TIC). Site teams from both ERIA and BEST will complete periodic self-assessments of degree of implementation using a Team Implementation Checklist (TIC), beginning at their first training session and continuing 2-3 times per year until they have achieved implementation with fidelity, and once per year thereafter. Site teams, which include teachers and administrative staff, complete the checklist together, either at a team meeting on-site or at a training event. The TIC provides a 3 point rating scale (Not Started=0, In Progress=1, and Achieved=2) of degree of implementation for the key elements of the practices, also serving to keep teams focused on the nature of those key elements.

The BEST/PBS TIC consists of 23 items in 7 areas for self-assessment of site level implementation, which provides an overall implementation score, as well as subscale scores in each of the key areas. The criterion for “implementation with fidelity” is an overall score of 80% plus 80% on the “expectations taught” item. It is expected that the majority of sites will take two years to achieve implementation with fidelity. Examples of items include: whether behavioral expectations have been defined and taught, whether rewards and consequences are in place, and whether data is gathered and used to monitor progress. The PBS/TIC is available through the work of the OSEP TA Center on Positive Behavioral Interventions and Supports (Sugai, Horner & Lewis-Palmer, 2001) and IVDB. (A sample TIC is in the Appendix.) For data entry and reporting, www.pbssurveys.org will be used. This web tool (developed and supported by the OSEP Center for PBIS as well) will enable sites to enter their TIC online and then see the results charted immediately. BEST cadre coaches will have passwords to access the reports on their schools sites, and thus use TIC results in the coaching process. SPDG evaluators will subscribe to www.pbseval.org which gives access to summary data on every school site. For each of their first two years of development, sites will also be encouraged to conduct staff surveys using a more extensive 50+ item Self-Assessment Survey, (Sugai, Horner & Todd, 2000), which reports what items are in place and which are priorities. This survey can be conducted and/or entered and charted on www.pbssurvey.org. (A sample Self-Assessment Survey is in the Appendix.)

The TIC for ERIA is under development and is being designed mirroring the PBS/TIC approach of rating items within key elements of practice implementation, drawing upon the Sopris West training manual used in ERIA, and in collaboration with ERIA Trainers and Coordinating Coaches and the Cohort 1 ERIA sites. Examples of items to be included are whether sites are conducting specific literacy skill assessments in decoding and fluency with struggling readers, whether students are placed into specific skill intervention programs based upon those assessments, whether reading intervention programs are being implemented with fidelity, and whether data on response to interventions is being used to determine next steps. Hard copies of completed ERIA/TICs will be submitted by sites to their coaches who will provide copies to SPDG evaluators. Eventually, a web-based tool will be developed.

2.3 Schoolwide Evaluation Tool (SET).  Fidelity of implementation will also be assessed by external coaches and SPDG evaluators using a Schoolwide Evaluation Tool (SET). In conjunction with site team self-assessment ratings through the TIC, SET provides for a multi-dimensional assessment which is facilitated by the SET items and scoring scheme paralleling those of the TIC. The SET is completed on-site through review of school data and documents, observation, and interviews. SETs will be completed annually for at least 20% of sites that finish the initial year of training and proceed to begin implementation. The correlation of externally conducted SET scores and internally generated TIC scores will be examined.

The PBIS/SET is an established research-validated instrument (Sugai, Lewis-Palmer, Todd, & Horner, 2001). (A sample SET is in the Appendix.) BEST SETs will be conducted by the sites’ cadre coaches and/or SPDG evaluators. As with scores from the TIC, the SET data can be entered into the www.pbssurvey.org web tool for charting and sharing of results with sites. SPDG evaluators gain access to the results through www.pbseval.org.

The SET for ERIA is under development and is being designed to mirror the BEST SET in terms of methodology and alignment with the TIC, while building in the elements unique to the literacy focus of ERIA. One important difference in the ERIA SET is that it will involve school principal and/or external coach observation of classroom teacher fidelity of implementation of the literacy interventions and pedagogy.

2.4 Student Outcomes Progress Monitoring.  Progress monitoring of ERIA site student English language arts (ELA) proficiency outcomes and BEST site office discipline referral and suspension/expulsion outcomes will be conducted by sites, coaches and the SPDG Evaluators. Outcomes data will be used as a formative assessment vehicle for feedback on progress and quality and will serve to continually focus personnel on the goal of improving student outcomes. For sites that have implemented with fidelity, outcomes data will also be used as a summative assessment of expected changed in performance measures.

The CST ELA will be a key measure for ERIA literacy outcomes monitoring for formative and summative assessments. Students are tested each spring, with test scores available the following fall. For the purposes of meeting the AMO benchmark of percent of students proficient, the critical factor in determining whether a school has made Annual Yearly Progress (AYP) under NCLB, students are considered “proficient” if they score in the Proficient or Advanced categories. The focus for ERIA sites is on narrowing the achievement gap between the ELA proficiency of students with disabilities and ELA proficiency for all students, while increasing the proficiency of the entire school population. An important additional progress monitoring data point with be tracking of potential decreases in the percent of students with disabilities scoring in the categories below Basic. For performance measurement, comparisons will be made between average CST scores from each school’s baseline year to scores from their second year of having fully implemented ERIA with fidelity. Annual performance measurement will take place in subsequent years to determine if their performance continues to improve.

CST ELA data are available from the CDE website in tables and research files. In addition, each school receives from the test contractors site level summary data and detailed data on each student, including cluster scores of reading comprehension and vocabulary items, which are used by sites as part of the ERIA individual student specific reading skill assessment process.

2.5 A CST Charting Program was designed by the SPDG Evaluation Team during SIG2 to enable school sites to create a single-page graphical chart in Excel showing the percent of all students compared with students with disabilities as a subgroup performing at the five levels of proficiency. While using the same numbers that parents, teachers, and administrators have been seeing for years, the program exhibits data with colors and proportions that makes it simple to grasp student progress with a glance and is particularly revealing of the achievement gap. A copy of the type of chart produced by the program is located in the Appendix.  ERIA sites (and all other SPDG sites, along with the CDE consultants) will continue to receive copies of the charting program, instructional movies on CD-Rom, and training in its use at statewide meetings.

Currently under development is a Site Academic Measures Metafile (SAMM), which is an extremely large database in which the SPDG Evaluation Team is compiling longitudinal CST, AYP and other demographic and academic performance data from a wide range of CDE databases and research files for all schools and districts in the state. This data is being integrated with TA utilization data from the SPDG. Pilot use of SAMM has enabled a preliminary look at the academic outcomes of sites that have accessed 3 or more days of Literacy or Collaboration TA as reported elsewhere in this application. The goal is to move toward a programmed format in which a several-page longitudinal data profile summary for a site could be produced by entering that site’s unique identifier code, as well as being able to do further outcomes-based research on the sites served through the SPDG, particularly with ERIA sites.

There is also a data tool being used by each ERIA site for tracking individual student assessment, placement, and reading intervention data, which facilitates site tracking of response to intervention and working closely with their coordinating coach in examining site-level data. (A sample ERIA individual student tracking tool is in the Appendix.) The possibility of setting up access to a web-based tool for tracking those data is being explored.

The BEST sites will be tracking Behavioral Incidence Data (BID), specifically office discipline referrals (ODR), suspensions, and expulsions. Each site develops its own tracking system or uses the approach mandated by their district. A few use the School-Wide Information System (SWIS), a web-based information system designed by the University of Oregon to help school personnel to use office referral data to design school-wide and individual student interventions. A summary BID form is submitted by each site to their cadre coach annually, and the data are forwarded to the SPDG Evaluators. (A summary BID form is in the Appendix.) For performance measurement, comparisons will be made between BID from each school’s baseline year and their second year of having fully implemented BEST with fidelity, with an expected decrease of 20% in suspensions/expulsions.

Supplemental information summarizing BID for the students with disabilities at each site will be obtained from the CDE California Special Education Management Information System (CASEMIS). In addition, the SPDG Evaluators will examine BEST site ELA data using SAMM to ascertain whether academic outcomes correlate with changes in behavioral outcomes at sites implementing BEST with fidelity. Again, all data is analyzed for use in continuous improvement.

ERIA and BEST outcomes data will be fed back to sites and reflected upon with sites at annual booster training and site facilitator meetings and teleconferences. Additionally, in consultation with external or cadre coaches, the data will be incorporated into a sustaining activity that assists sites in focusing on the progress they’re making in changing student outcomes as they implement scientific- or evidence-based instructional/behavioral practices. Summary reports for ERIA and BEST will be shared with the Evaluation Task Force, ISES, at Institute meetings, and through the CalSTAT website.

2.6 Evaluation of Coaches.  An online evaluation will be conducted annually to elicit feedback from the ERIA and BEST site cohorts on the work with their coaches to assess what’s working well, additional needs, what could be improved, and how it could be improved. SPDG evaluators will provide each coach with summary feedback to promote continuous improvement.

3. Evaluation Methods for the Leadership Community.  The SPDG training event evaluation process described earlier in this evaluation plan applies to all of the Regional Institutes and follow-up TA as well as Leadership Site TA, much of which supports and sustains ERIA and BEST practices. The Leadership Community itself is comprised of Leadership and Regional Host site team members who 1) participate in the annual State Leadership Institute, and 2) provide TA to one another and others to support systems change processes. For this group, the evaluation focus is quite different than in other activity areas. Here, the focus is on the development of a “knowledge strategy” for the professional development learning community (Wenger, 2004). The primary evaluation questions are: Are learning community members involved in translating their learning into useful practices for the community? What knowledge has the community produced in terms of lessons learned and best practices? Has that learning been “harvested” by the community and embodied in tools and other documents that enable it to share that knowledge within the community and with others? These questions will be explored using the following methods:

3.1 Development of a knowledge strategy and products.  During SIG2, a new development in the knowledge strategy of the Leadership Community of practice began to emerge. Case studies of the seventeen Leadership Sites were conducted, focusing on two areas. Interviews with each of the site teams focused on what and how systems change had occurred at their sites, with particular attention to special and general education collaboration. In another round, telephone interviews were conducted with a key informant from each site focused on what and how shifts in the use of resources occurred to make systems change possible.

The products of those interviews were individual site profiles which are now linked to the web pages for each Leadership Site. In this way, prospective TA recipients can get a sense of whether the challenges and changes experienced by that Leadership Site would be a good fit for the school/district seeking TA support for their own change process. In addition, cross site analyses were done for each set of data, articulating key commonalities and variations, and identifying critical factors and lessons learned. (A sample site profile and a report entitled, “They Are All Our Kids” based on the cross site analysesare located in the Appendix.) These profiles and reports were then fed back to the community online and face-to-face at the February 2007 State Leadership Institute. The discussion then centered on SPDG planning and “next steps,” meaning: 1) how these findings could be translated into a series of tools and documents, 2) how those tools and documents could then be refined, piloted, and eventually used and disseminated by the Learning Community in their ongoing transformations and 3) how the data could be used in their work supporting and sustaining other sites in making the systems change needed to successfully implement scientific or evidence-based practices. Potential tools and documents discussed included: a Readiness for Change Self-Assessment, a School Climate Survey, and a Collaborative Systems Change Self-Assessment. During SPDG, success in advancing the development of the Leadership Community knowledge strategy will be measured by the development, dissemination, and use of 2 or more new products that emerge from the knowledge base of the community. The tool and document development and refinement process will be documented; dissemination through hard copies and web downloads will be tracked; and use will be evaluated through user feedback forms and Institute roundtable discussions.

3.2 SAMM. The Site Academic Measures Metafile provides Leadership Sites and the Evaluator with longitudinal data and data profiles for measuring school site progress in student achievement. SAMM site profiles will be used by reviewers in the Leadership Site new and continuation application processes. In addition, summary data for each of the Leadership Sites will be presented in a single document at the State Leadership Institute so sites can assess their areas of weakness and arrange to receive TA from a site that is strong in that area.

4. Evaluation Methods for Parent/Family Involvement.  The SPDG will measure progress in facilitating parent/family involvement with schools and the broader educational system. The primary evaluation questions include: Are Family Participation Fund stipends being received by low income, under-represented ethnic minority parents/family members? Are they participating in both special- and general education-focused activities? Has their participation made a difference? Are they participating in the Leadership Community as team members? Do they have a significant role in deciding how the voices of parents are heard? These questions will be answered using the following instruments, data sources, and methods:

4.1 Family Participation Fund (FPF) Invoices/After Event Evaluations. Increases in the participation of low-income parents/family members, in parents attending general and special education decision-making committees and events, and in parents/family members reporting that their participation has made a difference will be measured using the FPF Invoice/After-Event forms. Parents/family members submit these forms following participation in a committee meeting or event in order to receive reimbursement; see sample in the Appendix. The form inquires about 1) income level and ethnicity to enable monitoring for receipt of stipends by the intended population of low-income, under-represented parents/family members; 2) the nature of the work of the committee (SE or GE), and 3) their effect through their presence and contributions (self-rated). Annually, FPF after-event evaluations will be forwarded by the FPF contractor to SPDG for data entry and summarizing into tables and graphs that will be shared with FPF staff and SPDG managers to monitor the use of the funds. These data will be summarized and included in the SPDG annual reports, and discussed at the Evaluation Task Force and ISES meetings.

            4.2 TED Event Registration Forms and Sign-In Sheets. The participation of parents/ family members as site team members at State Leadership Institute events will be measured using Institute team registration forms and sign-in sheets which ask participants to specify their role. This data is entered into TED. As event registrations come in, teams without parent members will be monitored, contacted, and requested to continue to seek a parent member who is able to attend. At the end of each annual event, summary reports will document the percent of State Institute teams with parent members. This data will be shared with the Evaluation Task Force, ISES, and included in the SIG annual report which is posted on the website.

4.3 ISES Task Force development of Parent-School Involvement Facilitation Survey. An ISES Task Force will be formed to assist and advise CDE in the development of data collection and interpretation processes for SPP Indicator #8 Parent Involvement, primarily through adaptation of the parent-school involvement facilitation survey instrument produced by the National Center for Special Education Accountability Monitoring (NCSEAM) for use in California. FPF funds will be used, in part, to support participation in this Task Force. The Task Force roster and agendas will be reviewed, and end-of-event evaluations will be tracked and fed back to facilitators. The products and processed developed will be documented as well.

5. Evaluation Methods for Teacher Recruitment, Preparation, and Retention. The evaluation questions include: Are more potential teachers accessing the www.teachcalifornia.org website? Are potential teachers taking advantage of online registration for tracking and support of their progress through the credentialing process? Is the enrollment of the CSULA Special Education Internship Program increasing? Are graduates of the CSULA program still teaching in special education three years after completing the program? These evaluation questions will be answered using the following data sources and methods:

5.1 Web usage tracking and online registration data. Software that enables website usage tracking will be sourced to examine the annual number of web visits, visitors, repeat visitors, page views, and downloads, as well as referral sources, and most accessed pages. Data from a web site user survey that appears as a pop-up will be analyzed to learn more about the characteristics of web visitors, including: age, gender, ethnicity, current status/interest in teaching, and areas of interest in teaching (special education, math, science, etc.). Analysis of data on the progress of online registrants through the teacher credentialing process will be conducted to better understand who is accessing the site and their progress toward credentialing. Annual increases in web traffic and user registrations will be expected following the first year of outreach. Use of the database by LEAs will be monitored. Data reports will be fed back to www.teachcalifornia.org outreach staff, IHE and Evaluation Task Force members, and ISES.

5.2 CSULA internship program enrollment, Individual Education Enrichment Plans, and graduate follow-up tracking. CSULA Special Education Internship Program enrollment figures will be tracked annually, with expected performance measure increases to begin after the initial year of outreach. Once in the program, interns complete and monitor, in collaboration with their support provider, an Individual Education Enrichment Plan (IEEP) in which they identify a professional goal to be achieved every 10 weeks throughout the school year. Support providers also conduct one formal observation of the intern every 10 weeks and complete an observation form that they share with the intern. Program graduates are contacted annually for three years following the two year program to assess retention in special education employment.

g.2. Evaluation examines the effectiveness of project implementation strategies. 

As shown in g.1 above, the implementation of SPDG strategies will be assessed for effectiveness through the monitoring of quantitative and qualitative data and against the performance measures. Issues around implementation of the activities will be documented by event evaluations, the 3-month follow-up surveys, team checklists, student outcomes analysis, interviews and observations. These will be addressed at the formative evaluation meetings.

g.3. Objective performance measures producing quantitative/qualitative data.  As shown in the narrative in g.1, the performance measures and proposed methods clearly produce both quantitative and qualitative data on performance pertaining directly to project objectives. The analysis plan for the evaluation is yet to be finalized and is dependent on the final structure of the data. In general, descriptive procedures and non-parametric analytical processes will be used, and where appropriate, parametric processes will be used. Quantitative data will be managed and analyzed using Filemaker Pro, Excel, and SPSS software. Qualitative data will be managed and analyzed using Atlas ti software.

g.4. Methods provide performance feedback and periodic assessment of progress.  The SPDG formative evaluation provides performance feedback at the activity level, to summarize data-to-date in terms of progress toward achieving outcomes, and to permit implementation adjustment based on findings twice within the project year with the Evaluation Task Force and ISES. The annual summative evaluation will then report on both project performance in completing the activities and progress toward achieving the intended outcomes. Formative and summative data and reports will be reviewed, with resultant recommendations made by the Evaluation Task Force and ISES.

Conclusion

            As California educators, we continue to search for ways to improve the system, to involve the community, and to better serve the children. In our efforts toward systemic change it is essential that we focus on how those changes affect the individual student, the unique child with disabilities. California’s Superintendent of Public Instruction Jack O’Connell in his State of Education address on February 5, 2007 stated: "Too often, the struggles of the African American student, the English learner, the learning disabled student were hidden by overall school achievement gains. That day is past. Today we are holding ourselves accountable for the results of all children. And when we see significant groups of students falling far short of the goal of proficiency that we hold for all students we must act. Today, equipped with specific knowledge of those gaps, we must focus as never before on solutions."  It is our most sincere desire that, in creating a better system, the SPDG will enable all children to succeed.

 

California Services for Technical Assistance and Training (CalSTAT)
A Special Project of the Napa County Office of Education| 5789 State Farm Drive, Rohnert Park, CA 94928
Fax: 707-586-2735 | email:info@calstat.org