The Oakland Unified School District has completed its third year of successful implementation of Urban Dreams - a five-year Technology Innovation Challenge Grant awarded by the United States Department of Education. Urban Dreams targets secondary students, teachers and families with a variety of activities designed to increase technology integration into social studies and language arts classes, improve student achievement, and increase technology access in the City of Oakland.
The project's major activities include:
 |
Ongoing professional development - The project staff has developed and implemented a comprehensive staff development program that is providing instructional and curriculum resources for teachers. This includes an initial two-week Summer Institute, a follow-up one-week Expository Writing Workshop, and monthly workshops. |
|
Classroom-based technical assistance - The project staff provides on-going technical support to teachers including in-classroom coaching, modeling, demonstrations and instructional assistance. |
|
Curriculum and other instructional materials - The project staff has purchased and trained staff on the use of several curriculum resources including source materials and databases. |
|
Technology access in the classroom – Urban Dreams staff has placed five computers and related technologies/software in the classroom of every Urban Dreams trained teacher. The district and project staff is providing ongoing technical assistance. The district staff is working to ensure high-speed Internet access in each classroom as well. |
|
Technology access in the homes of students - The project staff, in conjunction with its community partners, is placing desktop computers in the homes of some of its students. |
This year the project added a sixth major activity through supplemental grant funds made available by the U.S. Department of Education. The supplemental grant activity focused on developing digital video capabilities in teachers and students.
The project evaluation team consisting of project staff, evaluators and community partners created both a local evaluation plan and discrepancy evaluation model (DEM). The local evaluation plan outlined specific evaluation activities, timelines, and responsibilities. Major evaluation activities undertaken this last year include:
|
Collection and analysis of student data elements including standardized test results |
|
Development, administration, collection and analysis of the Student Technology Proficiency Inventory including the development of a representative comparison groups of project and non-project students |
|
Development, administration, collection and analysis of surveys and workshop evaluation forms for teachers and parents |
|
Development of interview and focus group interview protocols for project staff, teachers, parents and collaborating agency personnel; |
|
Focus groups and one-to-one interviews with project staff and teachers |
|
Development, administration, collection and analysis of a telephone interview protocol with families receiving computers; |
|
One-to-one interviews with directors from collaborating agencies |
|
Research and development of video case studies |
|
Development, administration, collection, and analysis of workshop evaluation forms |
|
Regular, on-site meetings (and other contacts) with project staff and teachers for data collection, technical assistance, and planning purposes; |
|
Analysis of district administered school technology survey |
|
Maintenance of an evaluation website |
|
Development and hosting of a two day evaluation and planning retreat |
|
Development and submission of a proposal for presentation at the 2002 American Educational Research Association. |
The DEM contains specific program narratives, component maps and input, process, and output (IPO) statements for each of the main program components (management, professional development, curriculum and community/parents) on three increasingly detailed levels. The DEM serves as a process evaluation instrument that provides stakeholders the means to examine the path of the project as it progresses though the five years of implementation. The DEM also contains the project’s evaluation questions. The local evaluation plan outlines general tasks, responsibilities and timelines.
The 2001-2002 academic year was the second full year of implementation. The staff and evaluators collected data on the Stanford Achievement Test-Ninth Edition (SAT/9) and STAR state proficiencies in English and social studies for a randomly selected cohort of Urban Dreams students and non-Urban Dreams students. This data demonstrates that students who participated in Urban Dreams classrooms scored significantly higher on both assessments than students who were not in Urban Dreams classroom.
The project undertook an extensive evaluation of the impact of the project on student technology proficiency. To do this, the evaluators developed an instrument to measure student technology proficiency based upon the International Society for Technology in Education (ISTE) standards. The instrument was given to the same randomly selected cohort of Urban Dreams students and non-Urban Dreams students. Students in the treatment group (i.e., whose teachers participated in UD) did report significantly more technology proficiency skills than the students in the comparison group after controlling for demographic, academic achievement/aspiration, and computer-specific background variables.
Professional development activities continued to be the
project’s primary focus during the third year.
Strong partnerships with agencies and universities aided in the design
and delivery of the professional development activities.
Professional development activities during the 2001-2002 academic year
included: a) a two-week Summer Institute for a new cohort of teachers; b) a
one-week expository writing and curriculum development workshop for continuing
teachers; c) on-going, core curriculum and instructional workshops; and d)
supplemental grant video technology workshops.
An evaluation of the 2001 Summer Institute was consistent with
previous years’ results showing a consensus among teachers that the institutes
were beneficial. Overwhelmingly,
participants rated the institutes’ presenters and activities as being of high
quality. Furthermore, participants
thought the resources provided them were of value and transferable to their
classrooms.
Participating teachers agreed that the weeklong expository writing and curriculum development workshop was beneficial and well executed. The project staff and evaluators created a lesson plan rubric that guided the lesson plan development by project teachers. Several teachers submitted lesson plans and other curricula materials that were subsequently published on the project’s website.
Teachers attended the project’s on-going professional development workshops on a voluntary basis during the 2001-2002 academic year. Participating teachers found the workshops of interest and benefit. Participation in the workshops varied primarily due to the demands teachers faced during the school year.
Urban Dreams’ Take-Home Computers program, in collaboration with the Marcus Foster Institute and the Oakland Technology Exchange (OTX), provided over 300 refurbished computers through the Home Computer Program during the 2001-2002 academic year.
The Marcus Foster Institute provided basic technology workshop
for parents receiving the computers. Just
over 300 parents attended one of the workshops that were offered at local high
schools. The
technology workshops provided basic computer concepts and skill development
through hands-on training. Parent
responses to the trainings have been overwhelmingly positive.
There were statistically significant (p<.01) gains by parents for
every competency measured by a pre- and post-assessment of basic computer
competencies.
A representative cross-section of adult participants in the Urban Dreams’ Take-Home Computer program participated in a telephone interview with project evaluators during the winter of 2001-2002. The goal of the interview was to evaluate the successes and limitations of the technology workshops and the Take-Home Computer program. The results of the telephone interviews revealed that adults in the families that received the computers believed the Take-Home Computer program was of worthwhile and were actively using their refurbished computers for a variety of activities. Ninety-two percent of the parents interviewed indicated they were able to go home and use the computer after they had received instruction in the technology workshops. Eighty-three percent of those homes interviewed were using the computer systems for homework. Perhaps most importantly, 78% of the respondents thought their child’s academic achievement or attendance had improved because of having access to a computer at home.
Though the basic goal of providing a basic computer into homes of needy families in the Oakland school district was achieved, the Home Computer Program still faced many challenges this year. Eighty-seven percent of the computers were not linked to the Internet. Over 45% of the families had at least some problems with their computers. Many respondents said the computers were slow, they lacked the necessary modem needed for Internet capability, and a CD-ROM was not included so they could be utilized with today’s larger programs. The project shared the results of the telephone survey with OTX and the Marcus Foster Institute. As a result, the project will offer this coming 2002-2003 academic year upgraded computer systems and a low-cost Internet option.
Urban Dreams continued to address the digital divide within the classroom. Each Urban Dreams teacher’s classroom was provided five computers, a presentation monitor, Internet connection, printer, laptop and access to language arts and social studies software. Teachers used these tools differently though were unanimous in their appreciation of them.
A full report providing a detailed description of the project’s progress toward meeting the stated goals and objectives is available through the Center for Evaluation and Research (CER) LLC. Included in the report are challenges (past and present), as well as recommendations and exemplars to further aid project staff with programmatic decision-making. More extensive progress evaluation reports and accompanying documents are available as well. Evaluators with CER are available to answer questions regarding this report. CER staff may be contacted via electronic mail at research@snowcrest.net or by phone at (530) 224-7704.
© Copyright 2002 Center for Evaluation and Research, LL