CalSTAT Technical Assistance and Training

WWW.TEACHCALIFORNIA.ORG  ANALYSIS

Prepared by Elaine R. Ader, PhD. For CalSTAT

 

EXECUTIVE SUMMARY

In Spring, 2010 a series of focus groups and surveys were administered to groups of students who reviewed the TEACHCalifornia.org website.  The goals of the site review were to determine whether TEACHCalifornia.org was intuitive in both its organization and navigation, to assess whether the site met the students’ needs, and to identify ways in which the site could be improved. The students ranged from high school students in Teacher Preparation ROP classes, to community college students in teacher education classes, undergraduates in teacher preparation programs, and graduate students in credential programs.  In total, 53 students were interviewed.  The interview questions focused on three major approaches to evaluating the TEACH California site:

• General Impressions – what they saw at “first glance”;
• “Just For You” – questions that they would want answered by a website on teacher preparation and jobs;
• Scavenger Hunt – looking for specific information on major topics.

In addition, focus group techniques were used to further explore the students’ impressions and interests.

The major highlights of the analysis are:

• The respondents overwhelmingly liked the site, regardless of the students’ background.  They found that the site provided “lots of information” and was well organized.
• The respondents liked the graphics and wanted more.
• The respondents used the navigation system to find the information they were interested in rather than utilizing site index or search features.
• They had a keen interest in finding jobs including
• Specialties that were hiring
• Areas of the state that are hiring
• “Getting a edge” on landing a job – writing resumes, taking specialized courses, knowing what likely interview questions are.
• They wanted information to be up to date – particularly as regards job availability and salaries/benefits.
• They had questions about credential programs.
• They were surprised to find that there was so much financial aid available.
• Most hadn’t heard of the site before participating in the study.
• Reactions to the site differed by population
• High School students focused on whether teaching was the right career for them and how to make that determination;
• Community College students focused on pathways to becoming a teacher;
• Undergraduates focused on credential programs and job prospects;
• Graduate students focused on lesson plans and the actual work of being a teacher.

 

SURVEY INSTRUMENT AND RESPONDENT GROUP

Survey Approaches

Two different approaches were used to gather information about the TEACH California website – a survey instrument and focus groups.  The survey utilized four approaches to finding out how students used the site and their degree of interest and satisfaction.  The “General Impressions” section was designed to get the “first impression” of the respondent.  The questions asked the respondents “in looking at the site for the first time” whether they found that it was easy to know where to start looking for information; whether the layout made them want to look around; whether they found the site interesting; and whether there was appropriate balance between graphics and information.

The “Just For You” section focused on the respondents’ priorities.  They were asked to choose three things that they would want to know about teacher training and finding a job that they would expect to find on the TEACH California site.  They were then queried on whether the topic they chose was addressed; whether it was easy to find the information; how they found the information; and whether they found the information interesting and accurate.

The “Scavenger Hunt” section was designed to determine whether respondents could locate specific pieces of information on the site; how they located the information (in particular how many “clicks” it took to find the information); and whether they found the information interesting and accurate.

The fourth section of the survey was designed to find out more about the respondents – whether they saw themselves as highly skilled web users; what their student status was; how they navigated the site (PC/MAC/mobile device); and what type of teacher they wanted to be.

The full survey instrument is located in Appendix A.

The focus groups asked for the same type of information but utilized the personal interview/group discussion format to explore answers more fully.  Additionally the focus groups were asked about how to publicize the site further and which search terms they used to find information on teaching. The focus group questions were:

• What caught your eye when first looking at the site?
• How easy was it to find useful information?
• What information was on the site that surprised you?
• Did you look around the site after you finished answering the questions?
• What do you think would be good ways to make this site known to other future teachers like yourself?
• Would you like to be able to ask more direct questions based on information on the site or information that you need (i.e., be able to interact with the site)?
• What search terms do you used in finding information about teaching?

The respondents who participated in the focus groups were given access to the TEACH California website 48 hours prior to their scheduled focus group so that their memory of the site would be fresh for the interview. In each of the question areas, the results from the focus groups corroborated the survey results.

Respondents

The respondents comprised of several different groups of students.

• Sixteen high school students in a teacher education ROP class took the survey and participated in the focus group. Two of those students did not intend to become teachers;
• Twenty nine community college students took the survey; nineteen of those students participated in focus groups.  Ten of the students were in online classes.  The community colleges represented included Sacramento City College, Golden West College, Coastline Community College, and Santa Ana College;
• Four students in the teacher preparation program at Sacramento State participated in the survey and focus groups;
• Four students in the teacher credentialing program at Sacramento State participated in the survey and focus groups.

In looking at the personal information that the respondents provided and the potential impact on how they used the website:

• Fifteen identified as “highly skilled” in navigating websites; Twenty nine identified as “moderately skilled”. While there was some concern that only the highly skilled would be able to navigate the site that did not prove to be the case.
• The overwhelming majority of respondents use PCs to access the site (forty one); fourteen respondents utilize Macs.  Nine respondents said that they utilized mobile phones to access the site.  While not a high number, it does suggest that website design should take account of the growing availability of mobile devices that access the web.  Several respondents indicated use of more than one device.
• There were a wide range of teaching interests expressed by the respondents, including:
• Special Education (12)
• Elementary/Multiple Subjects (37)
• Secondary (including specific subject areas) (20)
• Middle School (2)
• Teaching outside of the U.S. (2)
• Preschool (1)

 

FINDINGS

Focus Group Results

The primary findings from the focus groups are:

1. Respondents, regardless of background were impressed with all the information on the site, the categories, and the groupings.  Many of the respondents spontaneously said that they loved the site.  They all found useful information.  They thought the site was well organized.  One comment that summed up the general response was “Everything is at your fingertips”.  Several respondents mentioned the side panel menu.
2. They responded to the pictures.  Several respondents noted the “happy teachers”.  Several respondents (and this was borne out by the survey results as well) thought that there should be more graphics.  They liked the colors.
3. The information that surprised them included:
1. All the information
2. Amount of schooling needed to become a teacher (response from high school students)
3. Lists of school districts
4. Teaching information
5. Financial Aid information (but it was a “good surprise”)

4. Since many of the respondents were interested in finding jobs, they were particularly interested in the sidebar that described “Areas of Shortage” and the ability to look at jobs.  However one respondent suggested that instead of just outlining shortage areas there should be a map of California that indicates where different types of teachers are needed.  In general, this points to the desire to have updated information when it comes to jobs.
5. Several respondents (who were further along in their education) mentioned an interest in multiple credentials. Another respondent asked “What can you do with a bunch of credits in a subject?”
6. Several of the respondents have a English as a Second Language background and did not see information that addressed teaching English language learners.
7. In response to the question of whether they looked around the site after they finished answering the survey questions, the answers were largely “no”.  This could indicate several things: the survey had taken more time than they expected and they had other things to do, the site was not sufficiently intriguing to keep on exploring, and/or they would get back to the site when they had specific issues that they wanted to explore.  In general, the focus group responses left a sense that the site was now mentally “bookmarked” for the respondents so that if they had a question about teacher preparation and recruitment, they now knew that they had a resource.  However, it is difficult to assess the ultimate utility of the site until students need specific information which they either can, or cannot, find.
8. None of the focus group respondents were aware of the site prior to the interviews.  Several of the graduate students commented that they wish that they had been aware of the site earlier in their careers. When asked where they got their information if not from TEACH California, many of the respondents mentioned teachers and counselors.  Several utilized the CCTC and Department of Education sites but mentioned how difficult they were to navigate.  The undergraduates and graduate students were more likely to be aware of EdJoin.
9. In order to make others more aware of the site, respondents suggested providing the information to teachers and counselors.  Another suggestion was to include the information in the Sac State orientation.  When asked if there were specific search terms that they used to access information on teaching, the answers were not specific enough to redesign the site to capture the response (i.e, “math teacher”, “career as math teacher” or “requirements for math; “teaching”, “teacher salary”.  Only two respondents said “Teaching Requirements in California”).
10. When asked about whether they would like to interact with the site, several mentioned that they would like to contact a teacher in their field of interest or just have someone to email but most did not express an interest in that functionality. 
11. The students who were further along in their programs also mentioned being interested in lesson plans and teaching tools.  In particular, they thought that the “Distinguished Teachers” section was interesting but wanted to see more information on first year experiences, supports, and challenges as well as tips on classroom management and “words of wisdom”. The community college students mentioned being interested in seeing what a “Teaching Resume” would look like and thought that the category “Getting a Credential” was too broad to refer to them.
12. Two of the respondents mentioned that they thought the category “Out of Country Teacher” referred to how to get a job in another country and that they would be interested in that information.

In summarize, the overwhelming response to the site was positive.  The respondents found useful information, easy navigation, good organization, and a visually pleasing site.  Information that wasn’t available tended to focus on issues that were not mainstream – i.e., how to get multiple credentials or teach out of the country.  There is significant interest in up to date job information – a finding that is borne out by the surveys as well.

Survey Results

While the focus groups provided depth and the ability to follow up on specific responses to survey questions, the use of the survey instrument broadened the scope of the study and provided a range of responses.  The survey instrument measuring response to the TEACHCalifornia.org site was divided into three sections:

1. General Impressions – corresponding to the initial focus group question on first impressions
2. Just for You – corresponding to the focus group questions on individual priorities
3. Scavenger Hunt – providing a practical test of the site’s navigation and ability of respondents to find specific pieces of information.

General Impressions

The General Impressions section focused on the respondents’ reactions when first accessing the site.  When asked whether it was easy to know where to start looking around the site, the vast majority of respondents answered that it was easy.  Specifically of the sixteen high school respondents, all sixteen said that it was easy.  Twenty five of the community college students responded that it was easy.  Only three of the graduate and undergraduate students answered the questions in this section, and they agreed with this assessment as well.

When asked whether the layout made them want to look around, the response was somewhat mixed among the high school students but more positive for the college students.  Six of the high school students responded positively, seven said no, and three said “kinda”  Of the community college students, twenty had a positive response, three said “no”, and two said “kinda”.  Some of the comments were that the site was too plain and did not have eye appeal. Others said that they liked the colors.  The undergraduate and graduate students responded positively.

When asked if the site was interesting, the response was positive.  Nine of the high school students responded positively; while seven said “no” and two said that it didn’t stand out.  Twenty five community college students responded positively as did the undergraduate and graduate students.

When asked about the balance between graphics and information, there was  consensus that there was too much information and not enough graphics.  That was true among all groups although twelve of the community college students thought that there was a good balance.

Just For You

This section focused on the priority interests of the individual respondent.  The respondents were asked what the top three things were that they would want to know about teaching as a career and finding a job that they would expect to find on the site.  |The respondents cited a broad range of information they considered priorities that they would want to find on the TEACH California site, although many items grouped into categories   The responses varied somewhat by type of student but there were significant areas of overlap.  Table A summarizes the information that was requested.

TABLE A: RESPONSES TO THE TOP THREE INFORMATION ITEMS LOOKING FOR ON THE TEACH CALIFORNIA.ORG SITE

Information	HS	CC	Undergrad	Grad
Types of teaching	2
Salaries and benefits	3	5
Requirements – generally, multiple, and single	4	10	1
Requirements to become a substitute	1	1
Types of Teachers Needed	1
Do you need to major in the subject you wish to teach?	1
What college attend to become a teacher?	1
Types of degrees needed	1
Length of teacher training	1		1
What need to “get to” teacher training?	1
Scholarships	1	2
Teaching jobs outside CA	1	1
What grade is “best” for me	4
What is the “best” thing about teaching?	1
Is teaching right for me?	2
Where do I start?	1	1
How long will it take?	2
What qualifications need to have to get a job quickly?		1
Get you get experience before becoming a teacher?	2
Do you need to know what grade you want to teach before getting a credential?	2
How can you make sure you can get a job?	3	1
Can you do substitute teaching if you want to work part time?	1
What to do to get credentialed	4	1	3	1
Help finding a job	3		2	3
What grades need?	1
Credential programs?	2	4		1
Prerequisites to enter teaching programs		3	1
How to find programs for specific needs		1
Job search statistics		1
Substitute teaching to see if teaching right for them	1	1
How to write a portfolio		1
Programs, certificates, and resources for early childhood		1
Financial Aid		2
Basic things that schools look for in teachers		2
Costs		2
Lists of school districts participating in credentialing programs		1
CSET		1
Internship programs		1
How long does it take to find a job?		1
Where are there teacher shortages?		1
How do you become an out of country teacher?		1	1
What is FAFSA?		1
Where can you teach with a CA credential?		1
What else besides a credential do you need to teach?		1
Can you apply for jobs using the site?		1
How do you get tenured?		1
What is the difference between a school teacher and a special ed teacher		1
How do you prepare to get the most out of a teaching career?		1
How do you prepare a lesson plan?		1
How do you do time management in a classroom		1
How can you hurry up your program?		1
Will there be teaching jobs in the recession?		1
How do you create a teaching resume?		2
Current news regarding teaching		1
Classes need for a credential		1
Getting multiple credentials			1
English learner programs			1
Areas of state in high demand			1
Location of jobs and who to contact			1
Comparing and contrasting different training and schools			1
Special education info				1
Teaching strategies for reading, PBS, and autism				1
How to stay up to date with curricula				1

 

In looking at the responses by type of student, patterns emerge.  The high school students are very interested in how to find out whether teaching is the right career for them.  The community college students focus more on the nuts and bolts of the credentialing programs and finding a job.    The undergraduate students are interested in finding a job but also ask more specific questions about issues of personal interest – English learners or multiple credentials.  The responses from the graduate students begin to explore the mechanics of teaching and teaching in specific areas.

The second aspect of the “Just for You” section of the survey asked whether the information was easy to find.  Table B summarizes how many items the respondents could find on the site.

TABLE B: NUMBER OF ITEMS OF INTEREST FOUND ON TEACHCALIFORNIA.ORG SITE

Number Found	HS	CC	Undergrad	Grad
3 of 3	6	10	3	1
2 of 3	8	12	1	3
1 of 3	1	7	0	0
None	0	1	0	0

Among the information that the respondents said they couldn’t find was:

• List of districts participating in credentialing program
• How long it takes to find a job once a credential is obtained
• Where else you can teach with a CA credential
• How can you start if you get your degree in specific areas such as P.E. or psychology?
• How get tenured?
• How to make a lesson plan?
• Time management in the classroom
• Salary information by county
• How to become a substitute teacher after graduating college
• Earnings for elementary school teachers
• More news stories
• Scholarship information unclear
• Impact of additional training on getting a job
• Interview questions
• Whether to get multiple credentials simultaneously
• Teaching resume
• Teaching strategies
• Information on autism, PBS, reading
• Curricula to keep up with

Several of the questions imply evaluations that it is not possible to make on the site (how long it takes to get a job); other questions stem from a personal interest while others imply a broader focus for the TEACH California site.

In general, if the respondents found the information they were looking for, they found it to be accurate and helpful.

Scavenger Hunt

The final section of the survey asked the respondents to find three specific pieces of information on the site, describe what method they used to find the information, and, once found, whether the information was accurate and helpful.  While there were two sets of scavenger hunt questions asked, the second set had few respondents and will be omitted from the analysis.

The items that the respondents were asked to find were:

• Listing of school districts in California
• Credential programs in special education
• Special incentives for being a teacher in California

Overall, the respondents were able to find all three pieces of information.  They were able to use the site’s navigation system in order to find the information – often in 2-3 clicks.  They thought that the information was accurate and helpful –although several did not feel qualified to judge the information.  The one exception to this general finding was that nine of the high school students were not able to find the information on incentives (as compared with two community college students and one undergraduate).  The explanation for that may be that the high school students were not familiar with the terminology “incentives” and did not know where to search.

The summary of the responses is in Table C.

TABLE C: RESULTS FROM SURVEY SCAVENGER HUNT – NUMBER OF RESPONDENTS WHO FOUND INFORMATION

Information	High School	Community College	Undergrad	Graduate
School Districts/ Information Accurate (Yes)	15/15	19/19	4/4	4/2
Special Education Credential/Information Accurate Yes	15/14	19/18	3/3	4/4
Special Incentives/ Information Accurate	6/5	16/16	3/1	4/4

The navigation used for answering each of the questions varied but some of the paths included:

School District: Schools and places; find a school district; where teach; finding a job; site index – finding a job

Special Education Credential: Getting a credential for special education; understanding requirements – special education; types teaching; shortages; search feature; decide if teaching is for you; types teachers; getting a credential; understanding requirements; find the right program; “links to the right”

Incentives: Decide if teaching is for you (most respondents took this option); special education; find financial aid; search; site index; links to the right.  One respondent commented that s/he would like to find incentives for keeping people as teachers.

Site Analysis and Recommendations

In addition to the survey and focus group responses, the principal investigator reviewed the site and has several recommendations that incorporate both her impressions and respondents’ suggestions.

Consider formulating the site as three distinct but complementary parts

• Becoming a Teacher – this area which corresponds to the current top portion of the site focuses on the process of becoming a teacher – requirements, credential programs, supports.  Another aspect of the site that could be expanded would be a section on how to choose teaching as a career. The current information on the site assumes some teaching experience or access to a classroom. Several respondents mentioned trying substitute teaching to determine whether teaching was the right career – without understanding the level of training required to substitute teach.  In particular, high school and community college students do not have access to the type of information that can help them realistically assess teaching as a career with respect to heir aptitudes.  Information on skills and aptitudes and relevant volunteer work might be more meaningful to students who are first considering career choices.
• Getting hired – this area corresponds to the current right middle portion of the site and would include shortage areas, geographic areas that are hiring, and information/techniques for landing a job including creating a teacher resume, interview techniques, the availability of job fairs, and the use of substitute teaching to get a “foot in the door”.  If possible, this information should be as “real time” as possible.  Maps would be a helpful graphic for this section.
• Being a Teacher – This section corresponds to the area currently focusing on Distinguished Teachers. The area could expand to include lessons plans, tips on classroom management, and other “words of wisdom”.  Additionally there could be information on ongoing support for beginning teachers such as BTSA, financial incentives, or information on unions and teacher organizations.

While the section identifying different types of individuals who would be considering a teaching career, the category for ‘Getting a Credential” is too broad.  High school and community college students may not be familiar with that term.   A term like “Current Student” could allow for expansion into the categories of high school, community college, four year undergraduate, and graduate student.  Utilizing that approach, information targeted to those specific areas could be more fully developed.

While the site is designed to provide comprehensive information on teaching in California and to integrate information from a multitude of sources, the comments on emotional response that students had to the graphics of the smiling teacher, the interest in whether teaching is the right career, and the responses to the question on balance between graphics and text suggest that more graphics should be employed.  Pictures of job fairs, interactive maps showing hiring statistics, and videos of sample lessons could be very powerful in supplementing the strong information that is presented on the site.

APPENDIX A:  TEACH CALIFORNIA SURVEY

Thanks for agreeing to participate in this survey and subsequent focus group on the TEACH California website.  The survey has three parts:

• General Impressions
• Information Just for You
• Scavenger hunt

The survey should take you about 15-20 minutes.  The survey is in Microsoft Word. As you answer the questions, the text box for the questions will expand so that you can answer all the questions fully. In addition to answering the survey questions, you will also be participating in an online discussion group to discuss the website further.  Your professor will provide more details.

Thank you for participating in this project.  You will be helping future California teachers!

GENERAL IMPRESSIONS

As you look at the site for the first time,

1. Was it easy to know where to start looking around on the site?
2. Did the layout make you want to look around?
3. Was the site interesting? 
4. Was there balance between graphics and information?

JUST FOR YOU

As a Community college  student (fill in correct category),

What top three things would you want to know about teacher training and finding a job that you would expect to find on the TEACH California site? 

For the topic that you chose in item (a):     

• Was the topic addressed in the site (if the topic wasn’t addressed, put NA for the next questions)? ___Yes  ___ No  ______NA
• How easy was the information to find?
•  How did you find it? (What buttons did you click on/selections did you make?) 
• Once you found the information, was it accurate and helpful? 

(Repeat for topics 2 and 3)

SCAVENGER HUNT

There are some things that we would like you to “find” on the site.  The items are listed below.  For each item, please note:

• Did you find it?
• How did you find it?  What buttons did you click on/selections did you make?
• How many clicks did it take you to find the information?
• Once you found the information, was it accurate and helpful (to the best of your knowledge)?

SCAVENGER HUNT ITEMS:

1. Listing of school districts in California
2. Credential programs in special education
3. Special incentives for being a teacher in California

So that we can understand more about how you evaluated the site, could you provide us with additional pieces of information? 

Would you consider yourself:

1. Highly skilled in navigating websites?
2. Moderately skilled in navigating websites?

Not generally interested in navigating websites?

(Highlight one response)

What kind of teaching are you interested in (i.e. primary vs. secondary, subjects, special education…) List all that apply.   Primary.   2/3 grade. 

What technology would you use to access a website such as TEACH California? (highlight all that apply)

PC

Mac

Mobile device such as iPhone

Thanks again for participating in the TEACH California survey and focus group. 

 

 

 

 

 

 

California Services for Technical Assistance and Training (CalSTAT)
A Special Project of the Napa County Office of Education| 5789 State Farm Drive, Rohnert Park, CA 94928
Fax: 707-586-2735 | email:info@calstat.org