Urban Dreams Supplemental Grant Evaluation

Menu

Introduction

The Oakland Unified School District’s Urban Dreams Project requested a supplemental grant for the beginning of the third year of implementation.  Urban Dreams requested supplemental funding to expand activities related to the professional development and evaluation components of the original grant.  The activities were based on the project’s original objectives and address GPRA indicators.  The activities had the dual effect of immediately improving the quality of the project and providing for its long-term sustainability.  The following are the Urban Dream’s professional development objectives:

Objective 2.1  By June 2001 and annually thereafter, Oakland teachers who have participated in the Technology Integration Professional Development Program for at least two years will produce interdisciplinary, multicultural curricula reflecting the themes of human and civil rights.

Objective 2.2  By June 2002, 100% of social studies, language arts teachers and other staff members, as appropriate, working in project high schools will have completed year I staff education activities delineated in the “Technology Integration Professional Development Program.”

Objective 2.3  By June 2001 and annually thereafter, each cohort of Oakland teachers who have participated in the Technology Integration Professional Development Program for at least two years will demonstrate augmented content knowledge in the disciplines of social sciences and literature, technology uses in the classroom, and constructivist pedagogy as assessed through pre-, post- and ongoing professional development surveys, and through participant-developed curriculum units, thematic lessons and demonstration lessons

Urban Dreams Project teachers requested access and training related to digital video technologies.  Classroom teachers wanted to build on what they had learned through Urban Dreams to expand the use of video in their classrooms by both themselves and by the students they teach.  Urban Dreams requested supplemental funding to provide video cameras and video editing software at each of the project’s nine high schools for use by the project’s classroom teachers and their students.  The project requested funds for on-going training and technical assistance.

Supplemental Video Project

Urban Dreams contracted the services of Youth Media, an Oakland organization involved with the Oakland Public Schools and KDOL, the school’s television studio.  Youth Media (http://www.youth-media.org) states that it is “dedicated to training youth, educators and youth service providers in digital media and television production” (http://www.youth-media.org).  Youth Media selected teachers based on online applications submitted by Urban Dreams teachers.  (Appendix A: Urban Dreams Video Supplemental Application).  Youth Media conducted personal interviews with the teacher applicants as well in order to make the final selections.  (Appendix B: Teacher Interview Questions and Criteria for Selection).

Youth Media trained nine teachers and forty-five students on camera and equipment operations, script writing, storyboarding, and basics of nonlinear editing on Final Cut Pro.  Video workshop trainings were held on Saturdays and Sundays and teachers were paid a stipend to participate.  Nine teachers representing each of the Oakland High Schools participated in the video training workshops.  

In addition, a professional filmmaker selected by Youth Media was assigned to work with each teacher individually, as well as with the student film crew within the classroom setting.  (Appendix C: Filmmaker Interview Questions and Criteria for Selection)  The filmmakers were required to visit the classroom three times a week and assist the class with the completion of the final video product.  The teacher, five of their students and a Youth Media Filmmaker/Mentor worked with an English or History class to produce a five-minute video about Social Justice, Human and/or civil Rights. Videos are to be shown on KDOL TV, streamed on the web and disseminated on CD-ROM.

The provision of video technology provided the classroom teacher an additional medium for use in their curriculum projects (Objectives 2.1 and 2.3).  Each teacher, representing each of the Oakland high schools, received a Mac G4 computer with Microsoft Office 2001, Side by Side Scripting Software, and Final Cut Pro Video Editing Software.  In addition, each teacher received two Sony TRV 17 Digital Camcorders with cases, chargers, power supply, long life batteries. two tripods with cases, two microphones (one handheld, one lavaliere), cables (xlr/mini, firewire), and lens cleaning tissues.

Training Schedule for Urban Dreams Teacher Video Workshops

Language arts and social studies teachers used video to enhance their curriculum units.  Video technology allowed teachers to display the subject of human rights powerfully through visual media.  In addition to the teachers, students were given access to the video technology.  Students developed important competencies such as planning and collaborating with peers, writing storylines, creating sets/scenes, shooting video, downloading and creating video files and editing.  These high interest activities effectively engaged students in learning while they interacted with language arts and social studies content.  Final student products were scheduled for screening on September 9^th, 2002 at the Oakland Museum.  Videos topics included:

Teacher Evaluation of Video Supplemental Workshop Training (Appendix D)

The nine teachers who participated in the video workshops completed a survey on the final day of training, April 28, 2002.  The professional filmmakers had been assigned to work with the teachers and students and preliminary work had begun in most of the classrooms.  The scripts were in the process of being developed and the projects were due at the end of June, 2002.  Teachers were asked to give a short description of their video project and evaluate the training sessions of the workshop.  Responses indicated that all nine participants agreed that the video workshops were worthwhile, activities were logically sequenced, and that the staff provided adequate technical assistance.  All of the teachers felt that the technological and instructional resources were helpful and recommended this type of professional development to their colleagues.  All but one teacher reported that the filmmaker worked well with the students and that the collaboration with the teacher and filmmaker was effective.  This particular filmmaker did not continue with the project and another filmmaker was brought in to help finish that school video project.

When asked what could be done to improve the workshop training, three teachers had no recommendations.  Other teachers responded with:

N=9

Participating teachers were asked their opinion on the following questions regarding the effectiveness of video for teaching.  All nine teachers agreed or somewhat agreed that using video technology in classroom instruction is an effective strategy for teaching the subject of human and civil rights and engages students in learning English and/or Social Studies content.  One teacher remained neutral when asked whether student participation in the video project demonstrated improved student achievement while the other eight teachers agreed or somewhat agreed.  All but one teacher indicated that they plan to continue to incorporate video technology into their teaching. N=9

This impact extended to both the professional development of teachers and the academic achievement of students.  Teachers reported the following purposes for using video recording with their students. N=9

The teachers reported their current use of video recording with their students in the application prior to the project and again after the training.  Increased usage was reported after training. N=9

When asked what instructional strategies have been enhanced through the use of digital video technologies, teachers responded with:

N=9

N=9

One of the problems with this survey was that teachers completed them prior to the final session on editing.   It would be anticipated that the additional instruction and practice in the following session, as well as the experience gained in working with the professional filmmaker during the months after the training would result in reports demonstrating greater confidence and skill in the use of advanced video technology tools.  When asked what skills they gained through training in the use of technology, teachers responded with:

Additional comments made by teachers included:

Almost half of the teachers reported that they were at the most advanced stage of technology use at the time of the final workshop training session.  These teachers felt they were able to apply technology creatively to new contexts.  They were able to use technology as an instructional tool and integrate technology into curriculum. N=9

Additional materials, photographs, and information have been posted at the Urban Dreams Project evaluation website prepared by Center for Evaluation: http://ns1.californiaschools.net/~ud/webvideo/supplement/index.html.  A filmmaker’s journal provides greater insight into the work of the filmmaker in the classroom and is available at: http://ns1.californiaschools.net/~ud/webvideo/supplement/journal.htm

Student Evaluation of Video Project (Appendix E)

Students who participated in the video project as filmmakers were given an evaluation form by some of their teachers at the request of the evaluation team.  Twenty-three students responded to a student evaluation for the video project.  Most students indicated that they fell the time spent learning about video production was worthwhile for their future and that they gained skills in technology by participating in the video project.  Sixteen students agreed or somewhat agreed that being a part of the video production team improved their understanding of human and civil rights.

Students were asked, “What is the best part of being on the video production team?”  Their responses were:

Students were asked what they felt was the worst part about being on the video production team.  Responses included missing a fieldtrip, carrying the camera and equipment, missing class, staying after school, coming at lunch everyday, and having to do research.  Several students also indicated that they felt some pressure from the process.

Five students indicated that there was no negative aspect to being a part of the video production team.  They stated:

Video Case Studies

Directly tied to the use of video technology by teachers and students, was the request to create video case studies.  Employing instrumental case study methodology, the evaluation team provided an in-depth, contextualized review of what actually occurred in three of the classrooms.  Video case study methodology was ideally suited for understanding how technology was being integrated into the content areas.  Teachers and students provided video documentation of the activities that occurred in their classrooms.  This co-participatory role by teachers, students and evaluators increased teacher and student interest in the evaluation.  These case studies will be important sustainability tools to be used by district personnel as evidence of the program’s impact with internal and external funders.  Similarly, the case studies can be used by other urban districts as visual roadmaps for effective use of technology in classrooms.

In preparation for this step, the evaluation team prepared three case studies during the second year of the grant.  Three additional teacher case studies were conducted during the third year of the Urban Dreams Project.  Case studies for the second and third years of the grant are accessible on our web server that is dedicated to the dissemination of the Urban Dreams project at: http://californiaschools.net/ud/webvideo/vidindex.htm  (must have QuickTime to view video).  These case studies provided first-hand details and personal insights of teachers and students regarding the process of integrating technology into Language Arts and Social Studies instruction.  Teacher and student perceptions provide a richer, more detailed picture of the impact of the Urban Dreams project on learning, particularly as it relates to the themes of human and civil rights.

The video case study process went into great depth in the third year with on-going evaluation and interviewing of both students and teachers in three high school classrooms in Oakland:

The video supplemental training and the development of video case studies made an impact on the quality of the Urban Dreams project.  These activities helped Urban Dreams achieve the original scope of work and objectives. 

Appendix A: Urban Dreams Video Production

Teacher Pre-assessment

Personal Experience with Video

1.  Have you had access to a video camera for personal use?   Yes   No

2.  How often do you use a video camera for personal use?

 Frequently         Occasionally            Rarely                      Never

3.  Have you had previous instruction and/or experience in video production?

Yes   No

 If yes, please describe your experience with video production?  

4.  Have you had access to a digital video camera?  Yes   No

5.  Have you captured or digitized video on your computer?  Yes   No

If yes, please describe the equipment you used (Hardware and Software)

6.  Rate your familiarity with the following video recording technologies: (None or Never/Limited /Some/ Extensive)

7.  Rate your familiarity with the following video processes: (None or Never/Limited /Some/ Extensive)

Classroom Experiences with Video

8.  Have you had access to a video camera in your classroom prior to the Urban Dreams video supplementary grant?    Yes   No

9.  How often do you currently use video recording with your students?

 Frequently           Occasionally            Rarely                      Never

10.  For what purposes have you used video recording with your students in the past? 

11.   In what ways to you plan to utilize digital video in your classroom instruction?

12.  What instructional strategies to you currently use that you feel will be enhanced through the use of digital video technologies?

13.   What skills do you hope to gain through training in the use of digital video? 

Experiences with Technology and Urban Dreams

14.  Urban Dreams Cohort:     

 Cohort I, 1999-2000           Cohort II, 2000-2001

15.  Which Urban Dreams activities have you participated in?  Click all that apply.

16.  Please read the descriptions of each of the six stages related to the adoption of technology.  Check the stage that best describes your level at this time.

Stage 1: Awareness - I am aware that technology exists but have not used it, perhaps I'm even avoiding it.

Stage 2: Learning the process - I am currently trying to learn the basics.  I am sometimes frustrated using computers.  I lack confidence when using computers.

Stage 3: Understanding and application - I am beginning to understand the process of using technology and can think of specific ways in which to use it.

Stage 4: Familiarity and confidence - I am gaining a sense of confidence in using the computer for specific tasks.  I am starting to feel confident and comfortable.

 Stage 5: Adaptation to other contexts - I think about the computer as a tool to assist me and am no longer concerned about it as technology.  I use it in many applications and as an instructional aid.

Stage 6: Creative application to new contexts - I can apply what I know about technology in the classroom.  I am able to use it as an instructional tool and integrate it into the curriculum.

Appendix B: URBAN DREAMS VIDEO PROJECT INTERVIEWS

INTERVIEWER _________________                  DATE _____________________

APPLICANT _______________________SCHOOL _______________________

QUESTIONS

1.  Why are you interested in this project?

2.  What do you expect to get out of the process for yourself and for your students?

3.  How do you envision this working? I.e. your time with students, etc.

4.  Which class will produce a video and during what period? (confirm time of day including early release days)

5.  Are you willing to train on Saturdays (3-4 between now and June)

6.  Are you available for a training session on Saturday, February 23^rd 8:30-5:30?

7.  What are your criteria for selecting the five students that will serve as your primary production crew?

8.  What are your preliminary ideas for the project?

9.  How important is youth involvement in determining the direction of the piece?

10.  Production takes time management skills, give examples of projects you’ve done that require time management skills. (Student clubs, activities, etc.)

11.  What production experience have you had?

12.  What do you know about the process?

13.  How would you rate your capacity to learn new things?

Aesthetic-scriptwriting, storyboarding

Technical-Digital equipment

14.  What is your comfort level when a student knows more than you do?

15. Do you need to be in total control or do you have the ability to let go? Please explain?

16. Where will you store the equipment? (cameras, tripods) and how will you manage the equipment? (Security, check-out, use after hours)

SELECTION CRITERIA FOR URBAN DREAMS VIDEO PROJECT APPLICANTS

INTERVIEWER _______________________    DATE __________________

APPLICANT _________________________     SCHOOL ________________

PLEASE RATE FROM 1-10, 10 BEING THE HIGHEST SCORE

1)  Willingness to involve students, allow students to take ownership and provide opportunity for significant student involvement

2)  Good time management skills

3)  Strong sense for need to secure and care for equipment

4)  Preparation for interview

5)  Availability /Time commitment

6)  Open-mindedness

7) Ability to take constructive criticism

Appendix C: FILMMAKER INTERVIEWS

DATE:             ___________________                      INTERVIEWER:          ________

                                                                                                                                   (Initial)

FILMMAKER:   ___________________

1.  Tell us a little bit about yourself.

2.  Why are you interested in the project?

3.  Have you had previous teaching or training experience? In what capacity?

4.  Please discuss your experience in working Urban High School students.

5.  Please discuss you pre-production strategies.

6.  Please discuss the areas of video production/post-production expertise you are most comfortable with.

7.  Please discuss the areas of video production/post-production expertise you are less comfortable with.

8.  What types of production equipment are you familiar with?

9.  Please discuss your familiarity with non-linear editing in general and the Macintosh OS/Final Cut Pro in particular.

10. What is your weekly availability to participate in this program? When are you not available? Will you be in town on February 23^rd?

SELECTION CRITERIA FOR URBAN DREAMS FILMMAKER APPLICANTS

APPLICANT    _______________________                 DATE __________________

INTERVIEWER   _____________________

PLEASE RATE FROM 1-10, 10 BEING THE HIGHEST SCORE

Appendix D: Urban Dreams Teacher Survey

Video Supplemental Workshop Training

Your Name: ________________________School ____________________________

Video Project Name:_____________________________________________

Please provide a short description of your video project:

I.  TRAINING - Please check (Ö) the box that best describes your response:

A. My time spent in the video training workshop sessions was worthwhile.

Agree    Somewhat Agree    Neutral     Somewhat Disagree    Disagre

B. The activities were logically sequenced.

Agree    Somewhat Agree    Neutral     Somewhat Disagree    Disagree

C. The staff provided the technical assistance I needed.

Agree    Somewhat Agree    Neutral     Somewhat Disagree    Disagree

D. The technological and instructional resources were beneficial.

Agree    Somewhat Agree    Neutral     Somewhat Disagree    Disagree 

E. I would recommend this type of professional development to other colleagues.

Agree    Somewhat Agree    Neutral     Somewhat Disagree    Disagree

II.  Is there anything we could do to improve the workshop training?

III. TEACHING - Please check (Ö) the box that best describes your responses.

IV.  Please read the descriptions of each of the six stages related to the adoption of technology.  Check the stage that best describes your level at this time.

 Stage 1: Awareness - I am aware that technology exists but have not used it, perhaps I'm even avoiding it.

 Stage 2: Learning the process - I am currently trying to learn the basics.  I am sometimes frustrated using computers.  I lack confidence when using computers.

 Stage 3: Understanding and application - I am beginning to understand the process of using technology and can think of specific ways in which to use it.

 Stage 4: Familiarity and confidence - I am gaining a sense of confidence in using the computer for specific tasks.  I am starting to feel confident and comfortable.

  Stage 5: Adaptation to other contexts - I think about the computer as a tool to assist me and am no longer concerned about it as technology.  I use it in many applications and as an instructional aid.

Stage 6: Creative application to new contexts - I can apply what I know about technology in the classroom.  I am able to use it as an instructional tool and integrate it into the curriculum.

V. Additional Comments:

Appendix E: Urban Dreams

Student Survey for Video Project

Your Name: _________________________School ____________________________

Video Project Name: ___________________________________________________

I.  What is the purpose of your video project?

II. What is your job on the video production team?

III.  Please check (Ö) the box that best describes your response:

A. My time spent learning about video production was worthwhile for my future.

Agree    Somewhat Agree    Neutral     Somewhat Disagree    Disagree

B. I gained skills with technology by participating in the video project.

Agree    Somewhat Agree    Neutral     Somewhat Disagree    Disagree

C. Working on the video project helped me learn English and/or Social Studies.

Agree    Somewhat Agree    Neutral     Somewhat Disagree    Disagree

D. I improved my understanding of human and civil rights by being a part of the video production team.

Agree    Somewhat Agree    Neutral     Somewhat Disagree    Disagree  

E. Being on the video team made me want to come to school.

Agree    Somewhat Agree    Neutral     Somewhat Disagree    Disagree

IV. What is the best part of being on the video production team?

V.  What is the worst part about being on the video production team? ]