Type of contribution: Paper
Themes: networking the learner, flexible and distance
education, learner-centred teaching, innovative pedagogical methods,
ICT as a catalyst for change
Title: Virtual Portfolios: Lessons Learned from Four
Years of Implementation
Keywords: pedagogy, collaborative learning, higher
education, self-assessment, communication
Author's signed permission to present the contribution, email address
and web pages at the WCCE2001 Electronic Conference during the year
2001: Has been supplied already.
Authors:
Eugene S. Takle (gstakle@iastate.edu); Iowa State University;
International Institute of Theoretical and Applied Physics; Agronomy
Hall; Ames Iowa 50011 USA; Tel. (+1) 515-294-9871; Fax . (+1) 515-294-2619
http://www.public.iastate.edu/~wxintro/faculty/takle.html Elsebeth K.
Sorensen (eks@hum.auc.dk); Aalborg University; Dept. of Communication;
Kroghstraede 3; DK-9220 Aalborg Oest; Denmark; Tel. (+45) 9635 9077;
Fax. (+45) 9815 9434. http://www.hum.auc.dk/ansatte/profiler/es/index.html
Michael R. Taber (mrtaber@bentley.univnorthco.edu); Dept. of Earth Sciences;
University of Northern Colorado; Greeley, CO 80631; USA; Tel. (+1) 970-351-2470
Douglas Fils (fils@iastate.edu); Iowa State University; International
Institute of Theoretical and Applied Physics; Ames Iowa 50011 USA. Tel.
(+1) 515-294-6196; Fax. (+1) 515-294-9933
Abstract:
The Internet is being used more frequently for educational support,
not only in continuous and lifelong learning, but also as an alternative
method in higher education. The obvious advantages of this platform
include enhanced flexibility and time and space independent organization
of the learning process within higher education.
The availability, however, of this promising and flexible potential
as a platform for design and management of virtual learning processes,
does not guarantee its utilization, - or quality in the way it is utilized.
The need for a significantly enlarged range of design and delivery structures
has become evident. Implementation structures that support the learners'
navigation, interaction and collaboration at many levels in the virtual
world are both possible and essential. The virtual portfolio is an example
of one such structure that helps the organization of knowledge and knowledge
building for the learner in supporting the process of awareness and
the processes of (inter)actions in the virtual learning space. The personal
portfolio provides the student with an electronic home from which to
link to web course material, to link privately to other students in
collaborative learning experiences and to the instructor for assessment
and feedback in the learning process. Careful structuring through virtual
portfolios supports and adds quality to both virtual learning and virtual
instruction through enhanced overview of the learning process and content,
increased clarity of learning expectations, and individual and collaborative
spaces for learning activities and self-reflection. It also helps supporting
smooth flow and contextual organization of ideas, despite e.g. separate
and distinctly different historical educational traditions of participants.
This paper addresses the structuring potential of virtual portfolios
for web-based learning. It reports on 4 years of experience in using
this tool in a web-based American course on global environmental change,
as a means of creating and structuring collaborative and individual
spaces throughout the learning process. In one of our implementations
the virtual portfolio has been used collaboratively by a small group
to run numerical experiments of authentic simulation of plants interacting
with atmospheric and soil environments. Students share results of numerical
experiments and collectively (albeit asynchronously) analyze, interpret,
and report results. In this way, students act in roles of professional
researchers working as a research team, a time-honored method for creating
new knowledge. From our experience we have gained insight on how students
respond to different functional capabilities offered within the virtual
portfolio and to different requirements for evaluating student performance.
As a result of our aim at supporting collaborative learning among students,
we have explored the challenges implied in the establishment and maintenance
of peer discussion and interaction, as well as the instructional aspects
of judging the quality of such group interaction and dialogue to work
for collaborative knowledge building. The paper describes the experiences
from the perspectives of portfolio designers as well as learners and
instructors. The paper further reflects on and discusses more generally
the design and the instructional use of virtual portfolios in virtual
processes of learning, in relation to criteria of quality within theories
of collaborative learning.
Virtual Portfolios: Lessons Learned from Four Years of Implementation
Introduction
The Internet is being used more frequently for educational support,
not only in continuous and lifelong learning, but also as an alternative
method in higher education (Bates, 1999; Harasim, 1999). The obvious
advantages of this platform include enhanced flexibility and time and
space independent organization of the learning process within higher
education (Harasim, 1999).
The availability, however, of this promising and flexible potential
as a platform for design and management of virtual learning processes,
does not guarantee its utilization, - or quality in the way it is utilized.
The need for a significantly enlarged range of design and delivery structures
has become evident (Kaye, 1993; Collis, 1996). Implementation structures
that support the learners' navigation, interaction and collaboration
at many levels in the virtual world (Sorensen, 1997) are both possible
and essential. The virtual portfolio is an example of one such structure
that helps the organization of knowledge and knowledge building (Stahl,
1999) for the learner in supporting the process of awareness and the
processes of (inter)actions in the virtual learning space (Sorensen
et al, 2000).
This paper addresses the structuring potential of virtual portfolios
for web-based learning. It reports on 4 years of experience in using
this tool in a web-based American course on global environmental change,
as a means of creating and structuring collaborative and individual
spaces throughout the learning process.
In section 2 the virtual portfolio and its potential for supporting
web-based learning is described. We also provide a brief summary of
the course context in which we have implemented the portfolio. Section
3 provides a description of the various types of portfolios we have
used and section 4 gives some quantitative results documenting student
performance characteristics under different implementations. We summarize
our experiences in Section 5.
Definition and Context for Implementing Portfolio
This section provides our definition of the online portfolio as well
as a brief outline of the context in which it is implemented.
Definition
A general short definition of an online portfolio is: A structured image
over time of a process of development (Sorensen et al., 2000). In more
specific terms, an online learning portfolio may be viewed as a structured
collection of items and functionalities developed under a reflective
process that represents and demonstrates knowledge, skill, abilities,
personality, processes, and learning experiences that may be used to
serve various learning and professional purposes.
A virtual portfolio, therefore, may serve as a structuring tool for
managing a growing multitude of components of the learning processes.
It may have the form of an individual portfolio, built and functioning
from the perspective of the individual learner or the learning assistant
(instructor, teacher, mentor, curriculum advisor). Or it may be deployed
as a group portfolio for structuring the collaborative activities of
a group of learners (collaborating students in a course or team of researchers)
. Any instructional approach in learning can be implemented in the portfolio.
Consequently, the extent to which an online portfolio enhances learning
is, to a large extent, dependent on the implemented instructional approach
and its assumed criteria of quality in the learning process.
2.2. The Context of Experience: "Global Change"
The Global Change (GC) course has been taught for 12 years at Iowa State
University and has been on the web with interactive dialog since 1995.
The course addresses environmental changes of global and regional scale
that threaten to disturb and degrade natural and managed ecosystems
and ultimately human health and welfare (US Global Change Research Information
Office, 2001). Although presently an on-campus course, GC was established
with the vision that ultimately it should be delivered globally to address
environmental problems that span national and continental borders. This
would require engaging students asynchronously over the web from a wide
range of cultures, languages, and educational traditions, to search
for multi-national solutions to global environmental problems. Therefore,
the course was structured from its onset with a vision toward virtual
interactions in support of learning environments.
Student virtual portfolios, introduced in 1997 (Taber et al., 1997),
allowed students to better manage their interaction in and through the
course. An instructor portfolio introduced the same year allows the
instructor to more intensively interact with the student as guide in
the learning process. The course, including materials entered by the
instructor and materials and public discussion entered by students,
was structured to be an organically growing database that offers students
an increasingly rich body of learning resources for each subsequent
offering of the course.
3. The Use of Portfolios in Global Change
This sections provides a description of the individual student portfolio
as well as the group portfolio.
Individual Student Portfolios
Students manage their interaction with the course through their personal
portfolios (figure 1).
The portfolio has a calendar function that gives students the assignments
that are due each day and the record of their assignments submitted.
Quizzes they are required to take with each learning unit are accessed
from the portfolio. Students use personal portfolios to archive all
their electronic submissions, instructor's grades and comments, and
responses of other students, faculty, or others to electronic dialog
comments. Students' reviews of research papers are posted on the web
and linked where appropriate to learning narratives. A "message
of the day" allows the instructor to quickly communicate individually
with all students through their personal portfolios. From their individual
portfolios, students manage their interactions (a) publicly in the general
dialog, (b) privately with other students in group portfolios they share
with other members of small groups, and (c) privately with the instructor
through which they submit their self assessments and dialog with the
instructor on the evaluation and assessment processes.
The portfolio was divided into three blocks with each block having nearly
identical elements to be assessed (e.g., quizzes, response to ethical
question, response to a broad-topic question, self assessment of writing
quality, etc.). By observing student performance on three successive
and identical assessment blocks, the instructor can observe the student's
learning process as well as individual learning products.
Group Portfolios
The Global Change course has used two implementations of group portfolios,
serving two different purposes.
Group portfolio for collaborative knowledge building
The first type of group portfolio provides workspaces for small groups
of students to interact privately and write a common document. Teams
of 3-4 students were established to take notes of class discussion and
instructor comments during the regular 50-minute class meetings. These
students met electronically through a group portfolio and produced a
document summarizing the discussion, which subsequently was posted on
the web as a contribution to the growing database for that particular
topic. These portfolios were private to the group and even the instructor
did not have access to them. Only those students of the group that participated
in the construction of the online document are acknowledged on the web
as co-authors.
Group portfolio as laboratory for collaborative experiments
The second type of portfolio was used in conjunction with an experimental
laboratory for the course. This portfolio provided an electronic workspace
from which teams of students could collectively design and run experiments,
archive results and develop reports summarizing results from numerical
experiments of authentic simulation of plants interacting with atmospheric
and soil environments. For this implementation, teams of 3-4 students
were assigned portfolios for sharing results of numerical experiments
and collectively (albeit asynchronously) analyzing, interpreting, and
reporting results. In this way, students act in roles of professional
researchers working as a research team, a time-honored method for creating
new knowledge. The students met physically to get started on the experiments
and then managed their interaction remotely. Interpretation and report
writing were done as a team effort and submitted electronically.
Instructor Portfolio
An instructor portfolio was designed to enable the instructor to view
student records of quiz scores and dialog submitted under various topics.
The instructor had the option to view entries of all students for a
given assignment or chronological entries of a single student for all
assigned tasks. The latter form of display was particularly useful for
two reasons: an overview scan of the entire record gives a complete
picture of a particular student's contribution over the evaluation period.
This allows the instructor to see the range of topics on which the student
had written and gives a general overview of the student's writing capabilities
and interests. Secondly, the concatenation of all writing for the evaluation
period allows the instructor to observe progress in achieving stated
goals for the online dialog. The instructor can observe the impact of
recommendations to the student by having permanent and shared access
to "before" and "after" products of the student.
Space also was provided for instructor-student dialog on student self
assessment and instructor evaluation thereof.
Impact of Implementing Alternative Requirements through Portfolio
Student online dialog is a very important element of the GC course as
discussed in section 2. Virtual portfolios facilitate implementation
of alternative pedagogical strategies for the use of dialog. For instance,
when the GC course was first established on the web in 1995 with dialog
encouraged but only voluntary, the volume of comments was relatively
low and the quality rather superficial (with some notable exceptions).
When virtual portfolios were introduced in 1997 we began to put requirements
on the dialog such as a minimum number of entries and eventually a demonstration
of higher-order reasoning. Although new topics have been added over
time, a sufficiently large common set of topics provides a useful database
of student-generated materials for analysis of how students respond
to different learning opportunities and imposed conditions. Table 1
shows how the number of comments per student in the second 5-week period
of the course changed over the years in response to these changes in
pedagogy. Evidently the more stringent requirements stimulated a substantially
higher volume of comments.
Table 1. Number of on-campus students enrolled in the Global Change
course since 1995, mean number of comments in each learning unit (covering
material equivalent to one lecture), and mean number of comments in
one assessment cycle (covering 15 learning units).
Year 1995 1996 1997 1998 1999 2000
students 32 31 32 33 26 45
comments/student/learning unit 0.069 0.106 0.35 0.39 0.32 0.48
comments/student/assessment cycle 1.0 1.6 5.3 5.9 4.8 7.2
In the first two 5-week periods (assessment cycles) of the 2000 offering
of the course, students were struggling to learn to use the higher-order
reasoning requirements in their online dialog. By the third assessment
cycle, students had fully grasped the intent and characteristics of
these requirements and posted more and substantially higher quality
comments. We took a random sample of 10 comments from dialog for three
different years and made a subjective judgement of the quality of the
comments and also did a word count on each comment. Mean values shown
in Table 2 indicate an increase in quality and an increase by a factor
of 26 in the size of each comment. The use of three assessment cycles
within the virtual portfolio with all materials archived allows the
instructor in online counselling of individual students to point to
specific deficiencies and improvements or lack thereof from cycle to
cycle.
Table 2. Mean subjective evaluation of comment quality and mean number
of words per comment in 10 comments randomly drawn from dialog in three
different years in the Global Change course.
Year Quality (0-10) # words/comment
1995 4.4 88
1997 3.2 93
2000 5.3 2,505
For the course offered in 2000, students were required to post 5 comments
per cycle, including 3 responses to other student comments. Additionally,
they were required to elicit 3 comments from other students to receive
full credit. Table 3 shows that students went far beyond minimum requirements
on number of comments and somewhat above the minimum on other categories.
Table 3. Number of comments made per student, number of comments in
response to other student comments, and number of comments received
from other students in the third assessment cycle for the course offered
in 2000.
Mean # comments per student 9.3
Mean number of responses to other student comments 3.2
Mean number of responses received from other students 3.3
From these experiences we assert that the portfolio facilitates direct
implementation of desired pedagogies and adds pedagological value to
conventional teaching tools. It can be used to implement assessment
cycles to allow assessment of learning process as well as learning products
and promote pedagological goals such as good writing.
Summary
The personal portfolio provides the student with an electronic home
from which to link to web course material, to archive their own learning
products, to link privately to other students in collaborative learning
experiences and to the instructor for assessment and feedback in the
learning process. Careful structuring through virtual portfolios supports
and adds quality to both virtual learning and virtual instruction through
enhanced overview of the learning process and content, increased clarity
of learning expectations, and individual and collaborative spaces for
learning activities and self-reflection (Sorensen et al, 2000). It also
helps support smooth flow and contextual organization of ideas, despite
separate and distinctly different historical educational traditions
of participants.
From our experience we have gained insight on how students respond to
different functional capabilities offered within the virtual portfolio
and to different requirements for evaluating student performance. As
a result of our aim at supporting collaborative learning among students,
we have explored the challenges implied in the establishment and maintenance
of peer discussion and interaction, as well as the instructional aspects
of judging the quality of such group interaction and dialogue to work
for collaborative knowledge building.
References
Bates, A. W. (1999): "Managing Technological Change:
Strategies for Academic Leaders" San Francisco. Jossey Bass. 1999.
Collis, B. (1996): Tele-learning in a Digital World. International Thomson
Computer Press. 1996.
Harasim, L. (1999): The Virtual-U Field Trials: Lessons about Teaching
and Learning Online. 1999.
Kaye, A. R. (1992): Learning Together Apart. In Kaye, A. R. (ed.): Collaborative
Learning Through Computer Conferencing. NATO ASI Series, vol. 90. Springer-Verlag,
1992. (pp. 1-24)
Sorensen. E. K. (1997): Learning in Virtual Contexts. Navigation, Interaction,
and Collaboration. Ph.d.-afhandling. Aalborg University. Danmark. 1997
Sorensen, E. K., Takle, E. S., Taber, M. R., Fils, D. (2000): CSCL:
Structuring the Past, Present and Future Through Virtual Portfolios.
To be published in Dirckinck-Holmfeld, L. & Fibiger, B. (2000):
Learning in Virtual Environments
Stahl, G. (1999): Reflections on WebGuide: Seven Issues for the Next
Generation of Collaborative Knowledge-Building Environments. In C. M.
Hoadley and J. Roschelle (Eds.), Proceedings of the Computer Support
for Collaborative Learning (CSCL) 1999 Conference (pp. 600-610). Palo
Alto, CA: Stanford University. [Available from Lawrence Erlbaum Associates,
Mahwah, NJ]. http://kn.cilt.org/cscl99/A73/A73.HTM
Taber, M. R., E. S. Takle, and D. Fils (1997): Use of the internet for
student self-managed learning. Preprints, Sixth Symposium on Education.
American Meteorological Society, 2-7 February, Long Beach, CA.
US Global Change Research Information Office, 2001: http://www.gcrio.org/
