From the web page 
http://www.cast.org/publications/stsstudy/index.html

CENTER FOR APPLIED SPECIAL TECHNOLOGY

The Role of Online Communications in Schools: A National Study

Executive Summary

Table of Contents:

  * Overview
  * Online Use Is Growing in Schools
  * The Need for Measurable Results
  * Goals of the Study
  * Research Design
  * How Student Performance Was Evaluated
  * The Results
  * Successfully Implementing Online Communications in Schools
  * Summary
  * Obtaining copies


Overview

"The Role of Online Communications in Schools: A National Study"
demonstrates that students with online access perform better.
The study, conducted by CAST (Center for Applied Special
Technology), an independent research and development
organization, and sponsored by the Scholastic Network and
Council of the Great City Schools, isolates the impact of online
use and measures its effect on student learning in the
classroom. The study compared the work of 500 students in
fourth-grade and sixth-grade classes in 7 urban school districts
(Chicago, Dayton, Detroit, Memphis, Miami, Oakland, and
Washington DC) - half with online access and half without.

The results show significantly higher scores on measurements of
information management, communication, and presentation of ideas
for experimental groups with online access than for control
groups with no online access. It offers evidence that using
Scholastic Network and the Internet can help students become
independent, critical thinkers, able to find information,
organize and evaluate it, and then effectively express their new
knowledge and ideas in compelling ways.

Online Use Is Growing in Schools

Our nation's schools are rapidly acquiring both computers and
network infrastructures. Between 1989 and 1992, for example,
schools' inventory of computers increased by nearly 50%. In
addition, surveys have confirmed the increasing prevalence of
the Internet and other online services in schools. Currently,
the U.S. Department of Education reports that 35% of public
schools have access to the Internet, and an additional 14% more
have access to other wide-area networks such as CompuServe,
America Online and Prodigy (Office of Educational Research and
Improvement, 1995).

This trend has been encouraged by numerous educators and
government leaders, including President Clinton. In his State of
the Union speech in January 1996, the President pronounced:
"Every classroom in America must be connected to the information
superhighway... by the year 2000."

The Need for Measurable Results

Linking the nation's schools to the Internet is a widely held
goal. If we provide every student with access to online
resources; opportunities to engage with experts, mentors, and
peers; and supports to master real-world information and
communication skills, we will go a long way toward helping our
children succeed in today's society.

This endeavor requires an enormous investment. It is not just a
matter of hardware and wires, it requires a new way of teaching
and learning. It requires the support of the whole educational
community, from the government to superintendents, principals,
teachers, and parents.

To succeed in the workplace, children need to become
independent, critical thinkers. They must know where and how to
find information, organize it, evaluate it, and then effectively
express their new knowledge and ideas. They must work
cooperatively in teams. Online communications plays a vital role
in preparing children for their adult lives. As we fund and
implement these new communications technologies, it is important
to know whether they actually improve student performance. It is
also critically important to know what steps need to be taken in
the school, the district, and the community to support their
effective use in the classroom.

Goals of the Study

The study began with the premise that providing our nation¥s
schools with computers and Internet access, while necessary, is
not sufficient if students and teachers are to reap the benefits
of online communications. Current research suggests that new
technologies are associated with positive changes in learning
activity, but further controlled studies are needed to target
the impact of online use in the classroom for teachers and for
students. The goals of this study were: (1) to measure the
effects of online use on student learning including information
processing, communication, and presentation skills, and (2) to
gain insights into what it takes to use online communications
effectively in school.

Research Design

The study included approximately 500 fourth and sixth graders in
28 elementary and middle school classes from 7 large urban
districts across the United States: Chicago, Dayton, Detroit,
Memphis, Miami, Oakland and Washington, D.C. The population was
divided into two groups: 14 experimental classes (with online
access to Scholastic Network and the Internet) and 14 control
classes (without online access). In each city, two classes in
each of two schools were selected: an experimental and control
fourth grade class from one school, and an experimental and
control sixth grade class from the other school. Requirements
for city participation in the study were designed to encourage
district-level and building-based support for participating
teachers.

Both experimental and control groups carried out a common unit
of study based on the school's curriculum in conjunction with a
curricular framework, activities, and worksheets developed by
CAST for the study. Civil rights, a familiar interdisciplinary
topic for schools, was selected as the instructional focus for
the study. The unit was designed to be taught by teachers in
both experimental and control groups in their existing
classrooms. All classes were encouraged to use technology-based
resources such as multimedia reference materials and video tapes
for their projects. Both control and experimental classes were
advised to use computers whenever appropriate. Only the
experimental classes, however, were allowed to take part in
online resources, activities, and communication. In both groups,
the emphasis was on integrated learning, helping students to see
connections among people, their actions, and the real world.

The goal was for all classes to study civil rights according to
the preferences of their particular group, while participating
in a small set of activities common to everyone in the study.
Students engaged in three major learning activities:
communications (discussing civil rights issues with peers,
teachers, family members, neighbors, people in the community),
research (using a variety of media to explore and synthesize
information from multiple sources) and creation of a final
project. The project was chosen as the final product for the
study because it is commonly part of the curriculum in both
fourth and sixth grades, and it provides material for
performance assessment.

How Student Performance Was Evaluated

Student learning was measured by evaluating students' final
projects. These evaluations were based on 9 learning measures.
The assignment required students to demonstrate knowledge, apply
skills, and illustrate their thinking processes. Student
journals and worksheets provided a forum for reflection and
self-evaluation. The learning measures used to assess the final
projects were built into the worksheets so that students had
opportunities to work with them as they built their projects.

An independent evaluator, an experienced school teacher trained
in assessment methods, scored the projects. The evaluator had no
prior knowledge of either the study or any of the participating
schools and was given the mandate to score the projects fairly
and without bias. In order to ensure the integrity of the
scoring process, a CAST researcher trained the evaluator using
assessment consensus strategies. Both the CAST researcher and
the evaluator initially scored the same set of five projects,
compared their scores, and then discussed the evaluations each
had given. This process along with detailed explanations for
each of the 9 learning measures allowed the evaluator to develop
a consistent set of standards with which to score the remaining
projects.

As an additional means of control, over a quarter of the
projects were randomly selected (half from experimental classes,
half from control classes) and scored by a second external
evaluator. Correlations of the two external evaluators' scores
substantiate the consistency of the main evaluator's project
ratings.

Teachers submitted both the completed student projects and the
materials students used to develop their projects. This work was
evaluated on a four-point rating scale (0 to 3) according to the
following 9 learning measures:

  * Effective of presentation (e.g., interesting, informative,
    creative)
  * Effectiveness of stating a civil rights issue
  * Accuracy of information in relation to selected issue
  * Presentation of a full picture (e.g., who, what, when,
    where, why, and how)
  * Demonstration of insight into civil rights
  * Effectiveness of bringing together different points of view
  * Completeness
  * Organization
  * Demonstration of "best work" (e.g., well planned, neat,
    showing initiative)

Additional data on student performance and attitudes and teacher
behavior and attitudes were collected from pre-study and
post-study questionnaires, in-class observations, teacher
telephone interviews, and records of time spent online. Use of
standardized test scores was not appropriate, given that the
core intervention took place over a two-month period and
differences in standardized test scores cannot be expected
within such a short time frame.

The Results

Two main research questions underlie the investigations of the
study.
1. What is the impact of online use on student performance and
attitudes?
2. What is the impact of online use on teacher behavior and
attitudes?

Student performance and attitudes were clarified by looking at
the effects of online use on: a) student projects, b) student
perceptions as reported on questionnaires, and c) teacher
reports of student performance. Teacher behavior and attitudes
were clarified by looking at the effects of online use on: a)
teacher perceptions as reported on questionnaires, and b)
teacher reflections as reported in phone interviews.

Student Projects

Overall, students with access to Scholastic Network and the
Internet produced better projects than students without online
access. They received higher scores in all 9 learning measures.
The higher scores were statistically significant for 5 of the 9
measures. In particular, students with online access were more
effective in their ability to: present their work, state a civil
rights issue, present a full picture (who, what, when, where,
why, how), bring together different points of view, and produce
a complete project.

These results are particularly striking because both
experimental and control classes undertook the same projects and
both were encouraged to take advantage of computer tools and
resources in the Civil Rights Unit. Teacher reports and the
makeup of the projects submitted confirm that students in the
control group did use computers although they did not go online.
This targets online use as a key differentiating factor in
student performance within this study.

Text only version

Student Perceptions

The positive impact of online communications on student learning
is corroborated by analyses of student reports on pre-study and
post-study questionnaires. Mean ratings of selected items in the
pre-study questionnaire were compared with mean ratings of the
same items in the post-study questionnaire in order to determine
the change in students perceptions from the beginning to the end
of the study.

Students who used online access became more confident and
students without online access became less confident, over the
course of the study, in carrying out and presenting a research
project. Unrelated competencies, such as basic skills in
reading, writing, arithmetic, did not show similar boosts in
confidence.

The impact of online use on student performance is further
supported by student responses to questionnaire items rating the
frequency with which students used computers for different types
of school work. Students in the experimental group reported
significantly increased use of computers in four different areas
- gathering information, organizing and presenting information,
multimedia projects, and help with basic skills. Once students
with online access had completed their projects, they reported
an increase in how much they use the computer for the types of
activities that are most closely related to a project-based unit
of study and for which online use is well suited. In contrast,
there is a decrease in the control group's reports of using
computers for the same activities.

Text only version

Teacher Reports of Student Behavior

Telephone interviews with participating teachers at the end of
the study provide rich anecdotal evidence of greater learning
for students with online access. Teachers in the experimental
group reported that during the study their students found
information more quickly, drew resources from a larger number
sources in a wider variety of formats, and dealt with
information in ways that made the material more relevant to
their lives. Teachers felt that e-mail and message boards helped
their students learn from other students, teachers and the
community at large.

Changes in Teacher Behavior

Teacher reports on the pre-study and post-study questionnaires
provide information on changes in teachers' behavior. Due to the
low number of teachers (28) and missing data on many of the
items, results from analyses of the teacher questionnaires can
only suggest trends. They are presented in conjunction with
anecdotal reports from telephone interviews to lend insight into
possible effects of online use on teachers' behavior and
attitudes and to provoke further inquiry.

Data from the questionnaires on the amount of time teachers use
computers in school indicate differences between the
experimental and control groups. Even though all classes were
encouraged to use computers in the Civil Rights Unit, teachers
with online access appear to have increased their use (mean
change of .70) from the beginning to the end of the study,
whereas teachers without online access appear to have made no
change (mean change of .00).

Over the course of the study, teachers with online access
indicated more frequent use of computers with students for the
types of activities that are related to a project-based unit.
The experimental group had a sizably greater increase in use of
computers than the control group for gathering information,
multimedia projects, and creative expression. In contrast, the
control group had a sizably greater increase in use of computers
than the experimental group for skill reinforcement and reward
or motivation.

Although the small number of respondents limits the significance
of these findings, they suggest that online use changed
teachers' perceptions of the role of technology, and their own
behavior in using it with their students. The direction of
change for the experimental teachers was toward using technology
in the way that literate adults use computers - to enhance
performance directly in gathering, organizing, and presenting
information. The direction of change for control teachers was,
from an educational standpoint, more traditional. They increased
the use of computers as teaching devices - using them to teach
basic skills, or merely to reward other kinds of skill
development.

The telephone interviews give further insight into the effects
of online use on teacher behavior. Teachers in the experimental
group in contrast to the control group reported that they dealt
with a wider range of information and personally learned more
about the topic of civil rights. They attributed most of this
learning to having access to online resources; however, they
also reported acquiring more information from their students
than did the control teachers. Teachers with online access also
reported more positive interactions with parents than did
teachers without online access including: more effective teacher
conferences, more parents visiting the classroom, and
communication with parents online.

Changes in Teacher Attitudes

Questionnaire ratings provide insight into teachers' perceptions
of the building-based assistance they received during the study.
In contrast to control teachers, who reported increasing
satisfaction with support from administrators and technology
staff over the course of the study, the experimental teachers
reported decreasing satisfaction. Because they were being
challenged to learn how to do new things, they needed additional
support and did not find existing supports sufficient. At its
most positive, this is a problem of rising expectations. It is
when apprentices begin to learn most that they require the most
from their mentors.

Text only version

Successfully Implementing Online Communications in Schools

The study results offer insights into what it takes to
effectively use online communications in schools. In addition to
support at the school to insure that new technologies such as
online access are actually working, teachers need training and
ongoing support to help them use this technology effectively and
integrate it into their classroom teaching.

Curriculum and Teacher Supports

The Civil Rights Unit was designed to allow teachers the
flexibility to teach a unit of study that they most likely had
taught in the past and it provided teachers with suggestions for
activities and ways to help students organize their work. The
curriculum supports helped students and teachers develop
strategies for analyzing and synthesizing information. An
often-heard criticism of online use in classrooms is that the
amount of resources available can easily overwhelm students and
teachers, thereby doing more harm than good. The STS curricular
framework and worksheets offered strategies to help students
identify key elements of their topic, to organize the
information they found into categories, and to structure their
ideas into a compelling presentation for their project.

Scores on student projects, student and teacher questionnaire
responses, and teachers' interview statements, show that
curriculum and ongoing teacher supports combined with online use
had positive effects on student learning and on teacher
practice. Evaluations of student projects offer the most
significant support for the importance of a curriculum that can
adapt current practices to the inclusion of online activities
and resources. Many teachers reported in their phone interview
that the curriculum framework, lesson plans, activities, and
worksheets for the Civil Rights Unit were useful. In addition,
teachers in the experimental group relayed the importance of
online communications in supporting their teaching efforts. They
spoke about the help they received from peers and experts
online, the community-building interactions they had with other
teachers and community mentors, and the benefits of the wide
range of resources available on Scholastic Network and the
Internet. As a result, teachers themselves became engaged in
learning.

Summary

This study represents an important step in investigating whether
online use will lead to increased learning. In our world of
rising demands for literacy, and particularly for literacy that
extends beyond print literacy, we will need to engage both
students and their teachers in becoming independent, critical
thinkers. Students and teachers will need to become active
learners who can find information, organize it, evaluate it, and
then effectively express their knowledge and ideas. The results
of this study suggest that online use can be an important tool
for both.


The Role of Online Communications in Schools: A National Study

Conducted by:
CAST (Center for Applied Special Technology)

Project Director: Sari Follansbee
Project Coordinator: Niki Gilsdorf
Training and Support: Skip Stahl
Curriculum Design: Joan Dunfey, Susan Cohen
Research: Bart Pisha, Bob Hughes

For additional information, contact:
Sari Follansbee, Ed.D., Director of Curriculum

Obtaining copies:

  * Download this Executive Summary in RTF Format
  * Download a full copy of the report in RTF Format


                      Back to Publications

                                d

      Copyright 1996 CAST, Inc., Last updated: 7/10/98 DMC



----------
Table 1

Online Use Improved Student Learning
Project mean scores of experimental and control groups on 9
measures of learning

Experimental Group (n = 66)
Control Group (n = 38)

Student Project Learning Measures (Scale 0 to 3)

1. Effectiveness of presentation

Experimental Mean: 1.42
Control Mean: 1.11
Difference between Experimental and Control Means: .31(p t-Test
Probability: .0300

2. Effectiveness of stating a civil rights issue

Experimental Mean: 1.26
Control Mean: 1.00
Difference between Experimental and Control Means: .26(p t-Test
Probability: .0755

3. Accuracy of information in relation to selected issue

Experimental Mean: 1.70
Control Mean: 1.61
Difference between Experimental and Control Means: .09
t-Test Probability: .5174

4. Presentation of a full picture (who, what, when, where, why,
how)

Experimental Mean: 1.31
Control Mean: .87
Difference between Experimental and Control Means: .44(p t-Test
Probability: .0007

5. Demonstration of insight into civil rights

Experimental Mean: .82
Control Mean: .74
Difference between Experimental and Control Means: .08
t-Test Probability: .6336

6. Effectiveness of bringing together different points of view

Experimental Mean: .57
Control Mean: .29
Difference between Experimental and Control Means: .27(p t-Test
Probability: .0472

7. Completeness

Experimental Mean: 1.41
Control Mean: 1.18
Difference between Experimental and Control Means: .23(p t-Test
Probability: .0502

8. Organization

Experimental Mean: 1.27
Control Mean: 1.26
Difference between Experimental and Control Means: .01
t-Test Probability: .9556

9. Demonstration of "best work"

Experimental Mean: 1.95
Control Mean: 1.87
Difference between Experimental and Control Means: .08
t-Test Probability: .5735

Aggregate of mean scores

Experimental Mean: 11.73
Control Mean: 9.92
Difference between Experimental and Control Means: 1.81(p t-Test
Probability: .0749

Return to Report


----------
Table 2

Online Use Increased Students' Reports of Using Computers for
School Work
Changes in students' ratings between pre-study and post-study
questionnaires reflecting their frequency of using computers to
do different types of school work

Experimental Mean Difference (n approximately 154)
Control Mean Difference (n approximately 137)

Questionnaire Items on Student Use of Computers (Scale 0 to 3)

1. To help with basic skills (reading, writing, arithmetic)

Experimental Mean Difference: .28
Control Mean Difference: -.36
Difference between Experimental and Control Mean Differences:
.64 (p t-Test Probability: .0001

2. To gather information

Experimental Mean Difference: .60
Control Mean Difference: -.27
Difference between Experimental and Control Mean Differences:
.87 (p t-Test Probability: .0001

3. To organize and present information

Experimental Mean Difference: .25
Control Mean Difference: -.10
Difference between Experimental and Control Mean Differences:
.35 (p t-Test Probability: .0194

4. For writing

Experimental Mean Difference: .09
Control Mean Difference: .21
Difference between Experimental and Control Mean Differences:
-.12
t-Test Probability: .4399

5. To do multimedia projects (writing, pictures, sounds)

Experimental Mean Difference: .35
Control Mean Difference: -.21
Difference between Experimental and Control Mean Differences:
.56 (p t-Test Probability: .0002

6. To make pictures, cards, banners, signs, etc.

Experimental Mean Difference: -.06
Control Mean Difference: -.23
Difference between Experimental and Control Mean Differences: .17
t-Test Probability: .2526

7. As a reward for good behavior, finishing work etc.

Experimental Mean Difference: .01
Control Mean Difference: -.05
Difference between Experimental and Control Mean Differences .04
t-Test Probability: .6758

Return to Report


----------
Table 3

Online Use Affected Teachers' Perceptions of the Support They
Received
Changes in teachers' ratings between pre-study and post-study
questionnaires of the building-based support they received

Experimental Mean Difference (n approximately 9)
Control Mean Difference (n approximately 6)

Questionnaire Items on Building-based Support for Teachers
(Scale 0 to 3)

1. Administrative support availability

Experimental Mean Difference: -.50
Control Mean Difference: .67
Difference between Experimental and Control Mean Differences:
1.17 (p t-Test Probability: .0174

2. Administrative support effectiveness

Experimental Mean Difference: -.60
Control Mean Difference: .60
Difference between Experimental and Control Mean Differences:
1.20 (p t-Test Probability: .0373

3. Technology support availability

Experimental Mean Difference: -.13
Control Mean Difference: 1.00
Difference between Experimental and Control Mean Differences:
1.13 (p t-Test Probability: .0492

4. Technology support effectiveness

Experimental Mean Difference: -.13
Control Mean Difference: 1.00(p Difference between Experimental
and Control Mean Differences: 1.13 (p t-Test Probability: .0954

5. School staff development availability

Experimental Mean Difference: .10
Control Mean Difference: .86
Difference between Experimental and Control Mean Differences: .76
t-Test Probability: .2006

6. School staff development effectiveness

Experimental Mean Difference: -.30
Control Mean Difference: 1.00
Difference between Experimental and Control Mean Differences:
1.30 (p t-Test Probability: .0439

----------
End of Document