Oakland Unified School District

3. Technology Access and 4. Parent Involvement:

Objective 3 .1 Parent Involvement: By June 2001, 75% of program participants' parents and/or guardians will participate in at least one parent activity appropriate to their parenting situation.

Objective 3.2 Community Engagement: By June 2002, members from collaborating Institutes for Higher Education (IHEs) and identified community partners in business and/or social service agencies will participate in at least one school-community activity appropriate to improvement of instructional service delivery to students.

Objective 4.1 Equal Access to Technological Instructional Tools: By June 2004, every high school student in Oakland Unified School District will have consistent access to computer technology as instructional and learning tools in their social studies and language arts classes and classrooms.

Activities	Products	Timelines
Structured interviews with project staff and collaborating partners	Interview transcriptions and follow-up summaries	Spring 2003
Administration to principals of the STaR school-based technology survey	Analysis of system changes at the school level	Summer 2002, fall 2002 and spring 2003
Analysis of community-based technology trainings	Pre and post assessments and summative evaluation summaries	Spring 2002 and 2003 – three interim reports have been completed thus far (12-02)
Follow-up telephone interviews with families who have received refurbished computers.	Report on family usage of computers	Winter 2003 – Telephone interview begin in January of new families who received Windows-based computers

Other evaluation activities include: 1) assistance with the annual TICG performance report database; 2) direct contacts with teachers and project collaborators; 3) participation in Western Cluster meetings (New Orleans and Oakland); 4) preparation and submission of AERA report

[i] A quasi-experimental comparison design is appropriately used for estimating the impact of partial impact programs like Urban Dreams which is unable to randomly place students in treatment and control groups. Because of the past difficulty accessing student level data, the project has settled for an ex post design; the comparison group is being developed after the start of the program. The project evaluators are identifying those variables that may represent significant differences between the intervention and comparison group. The evaluators are creating a statistical representation of the overall relationships among the variables through the use of a multivariate statistical design.

[ii] Given that it is not within the evaluators’ power to randomly place students in to treatment or control groups (let alone politically or ethically acceptable), the evaluators have chosen to use a quasi-experimental comparison design when comparing student SAT/9 and graduation test scores. A quasi-experimental comparison design is appropriately used for estimating the impact of partial impact programs like Urban Dreams which is unable to randomly place students in treatment and control groups. Because of the past difficulty accessing student level data, the project has settled for an ex post design; the comparison group is being developed after the start of the program. The project evaluators are identifying those variables that may represent significant differences between the intervention and comparison group. The evaluators are creating a statistical representation of the overall relationships among the variables through the use of a multivariate statistical design.

Suggested 2003 Winter and Spring Evaluation Meeting Dates and Topics

Dates	Activities
January 7 10 AM to 12 Noon	Discussion of initial findings for CTAP testing
January 28 10 AM to 12 Noon	Presentation of findings of parent interviews
February 11 10 AM to 12 Noon	Presentation of CTAP findings
February 25 10 AM to 12 Noon	Planning for Oakland Western Cluster Conference
March 7 and 8	Western Cluster Meeting
March 25 10 AM to 12 Noon	Preview of the new student technology survey and teacher survey groupings
April 8 10 AM to 12 Noon	TBA
April 21 to 25	AERA National Conference (Chicago) – Presentation of Urban Dreams evaluation findings
May 13-14	Evaluation Planning Retreat

Activities	Products	Timelines
Access to the district QSP database in August with the 2002 SAT-9 results and high school exit exam results Creation of quasi-experimental design[i] Development of a comparison group of district students based on teacher participation in UD professional development Analysis of SAT/9 scores for 2002 and high school exit exam results	Matched score (pre/post) analysis of changes for UD students; comparison of UD students with other non project students in the district Comparison of student results looking at both teacher integration (self-reported on teacher survey)	July – September 2002 and 2003
Dissemination of student portfolio system – provide information to UD teachers to see if any teachers are willing to field test the portfolio system – choose 2 to 3 teachers with stipends to field test the system	Collections of student work as evidence of teachers’ best practices	This is made available to the project. We can let teachers know that the portfolio system exists. If there is interest in its use than the project can provide workshops on its use.
Collection of student work during teacher interviews and videotaping	Collections of student work as evidence of teachers’ best practices	Spring and Summer 2002
Refinement and second administration of the Student Technology Survey Creation of “quasi-experimental design[ii] Development of a comparison group of district students Analysis of student survey of technology skills Submission and conference presentation at AERA of Oakland tech and student testing results	Matched score (pre/post) analysis of changes for UD students; comparison of UD students with other non project students in the district; submission to academic journal and presentation	Spring - summer 2002 and 2003

Objective 2.1 Curriculum Development: By June 2001 and annually thereafter, Oakland teachers who have participated in the Technology Integration Professional Development Program for at least two years will produce interdisciplinary, multicultural curricula reflecting the cultures and languages of Oakland’s student body through the themes of human and civil rights. Objective 2.2 Staff Development II: By June 2002, 100% of social studies, language arts teachers and other staff members, as appropriate, working in project high schools will have completed year I staff education activities delineated in the “Technology Integration Professional Development Program.” Objective 2.3 Staff Development I: By June 2001 and annually thereafter, each cohort of Oakland teachers who have participated in the Technology Integration Professional Development Program for at least two years will demonstrate augmented content knowledge in the disciplines of social sciences and literature, technology uses in the classroom, and constructivist pedagogy as assessed through pre-, post- and ongoing professional development surveys, and through participant-developed curriculum units, thematic lessons and demonstration lessons.
Activities	Products	Timelines
Analysis of CTAP survey Development of comparison groups using CTAP online system and comparison analysis of UD vs. non-UD teachers Analysis of survey results	Interim reports on teacher knowledge, attitudes and behaviors Matched score (pre/post) analysis of changes for UD teachers; comparison of UD students with other non project teachers in the district	Teachers began taking the CTAP survey online in the fall of 2002. CER staff has been granted administrative access to CTAP survey results and is developing comparison groups. Analysis will take place after most teachers have submitted their surveys.
Refinement and posting of an assessment rubric for teacher developed curricula plans/subsequent product emerging from the expository writing workshops	Assessment rubric and quality curricula posted on project website	Completed
Review of curricula and instructional materials developed by participating teachers	Collection of curriculum and instructional materials that illustrate promising practices	Ongoing
Follow-up interviews with teachers in the 2001-2002 supplemental grant video program	Measurement of the impact of supplemental grant video training	Baseline taken prior to video training and in the spring of 2002; follow-up interviews in the spring of 2003
Pre/post assessment of project sponsored workshops	Measurement of the participant attitudes and perceived benefit of project sponsored workshops	During summer institute trainings – interim report was finished 12-02

Oakland Unified School District’s Urban Dreams Technology Challenge Grant

Activities

Activities

Activities