Urban Dreams Supplemental Grant Evaluation

Menu

Introduction

The Oakland Unified School District’s Urban Dreams Project requested a supplemental grant for the beginning of the third year of implementation.  Urban Dreams requested supplemental funding to expand activities related to the professional development and evaluation components of the original grant.  The activities were based on the project’s original objectives and address GPRA indicators.  The activities had the dual effect of immediately improving the quality of the project and providing for its long-term sustainability.  The following are the Urban Dream’s professional development objectives:

Objective 2.1  By June 2001 and annually thereafter, Oakland teachers who have participated in the Technology Integration Professional Development Program for at least two years will produce interdisciplinary, multicultural curricula reflecting the themes of human and civil rights.

Objective 2.2  By June 2002, 100% of social studies, language arts teachers and other staff members, as appropriate, working in project high schools will have completed year I staff education activities delineated in the “Technology Integration Professional Development Program.”

Objective 2.3  By June 2001 and annually thereafter, each cohort of Oakland teachers who have participated in the Technology Integration Professional Development Program for at least two years will demonstrate augmented content knowledge in the disciplines of social sciences and literature, technology uses in the classroom, and constructivist pedagogy as assessed through pre-, post- and ongoing professional development surveys, and through participant-developed curriculum units, thematic lessons and demonstration lessons

Urban Dreams Project teachers requested access and training related to digital video technologies.  Classroom teachers wanted to build on what they had learned through Urban Dreams to expand the use of video in their classrooms by both themselves and by the students they teach.  Urban Dreams requested supplemental funding to provide video cameras and video editing software at each of the project’s nine high schools for use by the project’s classroom teachers and their students.  The project requested funds for on-going training and technical assistance.

Supplemental Video Project

Urban Dreams contracted the services of Youth Media, an Oakland organization involved with the Oakland Public Schools and KDOL, the school’s television studio.  Youth Media (http://www.youth-media.org) states that it is “dedicated to training youth, educators and youth service providers in digital media and television production” (http://www.youth-media.org).  Youth Media selected teachers based on online applications submitted by Urban Dreams teachers.  (Appendix A: Urban Dreams Video Supplemental Application).  Youth Media conducted personal interviews with the teacher applicants as well in order to make the final selections.  (Appendix B: Teacher Interview Questions and Criteria for Selection).

Youth Media trained nine teachers and forty-five students on camera and equipment operations, script writing, storyboarding, and basics of nonlinear editing on Final Cut Pro.  Video workshop trainings were held on Saturdays and Sundays and teachers were paid a stipend to participate.  Nine teachers representing each of the Oakland High Schools participated in the video training workshops.  

In addition, a professional filmmaker selected by Youth Media was assigned to work with each teacher individually, as well as with the student film crew within the classroom setting.  (Appendix C: Filmmaker Interview Questions and Criteria for Selection)  The filmmakers were required to visit the classroom three times a week and assist the class with the completion of the final video product.  The teacher, five of their students and a Youth Media Filmmaker/Mentor worked with an English or History class to produce a five-minute video about Social Justice, Human and/or civil Rights. Videos are to be shown on KDOL TV, streamed on the web and disseminated on CD-ROM.

The provision of video technology provided the classroom teacher an additional medium for use in their curriculum projects (Objectives 2.1 and 2.3).  Each teacher, representing each of the Oakland high schools, received a Mac G4 computer with Microsoft Office 2001, Side by Side Scripting Software, and Final Cut Pro Video Editing Software.  In addition, each teacher received two Sony TRV 17 Digital Camcorders with cases, chargers, power supply, long life batteries. two tripods with cases, two microphones (one handheld, one lavaliere), cables (xlr/mini, firewire), and lens cleaning tissues.

Training Schedule for Urban Dreams Teacher Video Workshops

Language arts and social studies teachers used video to enhance their curriculum units.  Video technology allowed teachers to display the subject of human rights powerfully through visual media.  In addition to the teachers, students were given access to the video technology.  Students developed important competencies such as planning and collaborating with peers, writing storylines, creating sets/scenes, shooting video, downloading and creating video files and editing.  These high interest activities effectively engaged students in learning while they interacted with language arts and social studies content.  Final student products were scheduled for screening on September 9^th, 2002 at the Oakland Museum.  Videos topics included:

Teacher Evaluation of Video Supplemental Workshop Training (Appendix D)

The nine teachers who participated in the video workshops completed a survey on the final day of training, April 28, 2002.  The professional filmmakers had been assigned to work with the teachers and students and preliminary work had begun in most of the classrooms.  The scripts were in the process of being developed and the projects were due at the end of June, 2002.  Teachers were asked to give a short description of their video project and evaluate the training sessions of the workshop.  Responses indicated that all nine participants agreed that the video workshops were worthwhile, activities were logically sequenced, and that the staff provided adequate technical assistance.  All of the teachers felt that the technological and instructional resources were helpful and recommended this type of professional development to their colleagues.  All but one teacher reported that the filmmaker worked well with the students and that the collaboration with the teacher and filmmaker was effective.  This particular filmmaker did not continue with the project and another filmmaker was brought in to help finish that school video project.

When asked what could be done to improve the workshop training, three teachers had no recommendations.  Other teachers responded with:

N=9

Participating teachers were asked their opinion on the following questions regarding the effectiveness of video for teaching.  All nine teachers agreed or somewhat agreed that using video technology in classroom instruction is an effective strategy for teaching the subject of human and civil rights and engages students in learning English and/or Social Studies content.  One teacher remained neutral when asked whether student participation in the video project demonstrated improved student achievement while the other eight teachers agreed or somewhat agreed.  All but one teacher indicated that they plan to continue to incorporate video technology into their teaching. N=9

This impact extended to both the professional development of teachers and the academic achievement of students.  Teachers reported the following purposes for using video recording with their students. N=9

The teachers reported their current use of video recording with their students in the application prior to the project and again after the training.  Increased usage was reported after training. N=9

When asked what instructional strategies have been enhanced through the use of digital video technologies, teachers responded with:

N=9

N=9

One of the problems with this survey was that teachers completed them prior to the final session on editing.   It would be anticipated that the additional instruction and practice in the following session, as well as the experience gained in working with the professional filmmaker during the months after the training would result in reports demonstrating greater confidence and skill in the use of advanced video technology tools.  When asked what skills they gained through training in the use of technology, teachers responded with:

Additional comments made by teachers included:

Almost half of the teachers reported that they were at the most advanced stage of technology use at the time of the final workshop training session.  These teachers felt they were able to apply technology creatively to new contexts.  They were able to use technology as an instructional tool and integrate technology into curriculum. N=9

Additional materials, photographs, and information have been posted at the Urban Dreams Project evaluation website prepared by Center for Evaluation: http://ns1.californiaschools.net/~ud/webvideo/supplement/index.html.  A filmmaker’s journal provides greater insight into the work of the filmmaker in the classroom and is available at: http://ns1.californiaschools.net/~ud/webvideo/supplement/journal.htm

Student Evaluation of Video Project (Appendix E)

Students who participated in the video project as filmmakers were given an evaluation form by some of their teachers at the request of the evaluation team.  Twenty-three students responded to a student evaluation for the video project.  Most students indicated that they fell the time spent learning about video production was worthwhile for their future and that they gained skills in technology by participating in the video project.  Sixteen students agreed or somewhat agreed that being a part of the video production team improved their understanding of human and civil rights.

Students were asked, “What is the best part of being on the video production team?”  Their responses were:

Students were asked what they felt was the worst part about being on the video production team.  Responses included missing a fieldtrip, carrying the camera and equipment, missing class, staying after school, coming at lunch everyday, and having to do research.  Several students also indicated that they felt some pressure from the process.

Five students indicated that there was no negative aspect to being a part of the video production team.  They stated:

Video Case Studies

Directly tied to the use of video technology by teachers and students, was the request to create video case studies.  Employing instrumental case study methodology, the evaluation team provided an in-depth, contextualized review of what actually occurred in three of the classrooms.  Video case study methodology was ideally suited for understanding how technology was being integrated into the content areas.  Teachers and students provided video documentation of the activities that occurred in their classrooms.  This co-participatory role by teachers, students and evaluators increased teacher and student interest in the evaluation.  These case studies will be important sustainability tools to be used by district personnel as evidence of the program’s impact with internal and external funders.  Similarly, the case studies can be used by other urban districts as visual roadmaps for effective use of technology in classrooms.

In preparation for this step, the evaluation team prepared three case studies during the second year of the grant.  Three additional teacher case studies were conducted during the third year of the Urban Dreams Project.  Case studies for the second and third years of the grant are accessible on our web server that is dedicated to the dissemination of the Urban Dreams project at: http://californiaschools.net/ud/webvideo/vidindex.htm  (must have QuickTime to view video).  These case studies provided first-hand details and personal insights of teachers and students regarding the process of integrating technology into Language Arts and Social Studies instruction.  Teacher and student perceptions provide a richer, more detailed picture of the impact of the Urban Dreams project on learning, particularly as it relates to the themes of human and civil rights.

The video case study process went into great depth in the third year with on-going evaluation and interviewing of both students and teachers in three high school classrooms in Oakland:

The video supplemental training and the development of video case studies made an impact on the quality of the Urban Dreams project.  These activities helped Urban Dreams achieve the original scope of work and objectives. 

Appendix A: Urban Dreams Video Production

Teacher Pre-assessment

Personal Experience with Video

1.  Have you had access to a video camera for personal use?   Yes   No

2.  How often do you use a video camera for personal use?

 Frequently         Occasionally            Rarely                      Never

3.  Have you had previous instruction and/or experience in video production?

Yes   No

 If yes, please describe your experience with video production?  

4.  Have you had access to a digital video camera?  Yes   No

5.  Have you captured or digitized video on your computer?  Yes   No

If yes, please describe the equipment you used (Hardware and Software)

6.  Rate your familiarity with the following video recording technologies: (None or Never/Limited /Some/ Extensive)

7.  Rate your familiarity with the following video processes: (None or Never/Limited /Some/ Extensive)

Classroom Experiences with Video

8.  Have you had access to a video camera in your classroom prior to the Urban Dreams video supplementary grant?    Yes   No

9.  How often do you currently use video recording with your students?

 Frequently           Occasionally            Rarely                      Never

10.  For what purposes have you used video recording with your students in the past?