Impacts of putting requirements on dialog in Distributed CSCL

Elsebeth K. Sorensen (eks@hum.auc.dk)
Aalborg University; Dept. of Communication; Kroghstraede 3;
DK-9220 Aalborg Oest; Denmark;.Tel. (+45) 9635 9077; Fax. (+45) 9815 9434

Eugene S. Takle (gstakle@iastate.edu)
Iowa State University; International Institute of Theoretical and Applied Physics;
Agronomy Hall; Ames Iowa 50011 USA; Tel. (+1) 515-294-9871; Fax. (+1) 515-294-2619

Abstract
Web-based learning is assuming an increasing role in education at college and pre-college levels. It is not only situations of continuing education and lifelong learning which call for flexible organization of the learning process, e.g. with the purpose of bridging distances. The obvious advantages in terms of a more flexible, time and space independent organization of the learning process, also within higher education, are clear. The flexible organizational design of web-based learning, by itself, does not make this method superior to other methods. Of more importance is the quality and design of the learning process. Many flexible designs offer no possibilities for interaction and collaboration among students, but are rather sterile and lack a framework to stimulate collaborative knowledge building. Others do so, but have problems - even when interaction occurs — in qualifying a knowledge building dialogue. This paper is dealing with the question of how to qualify a knowledge building dialogue through through grading requirements and as an effect from providing the students with meta-awareness around the functions of the requested comments in the dialogue. The paper suggests that grading requirements as well as providing awareness around the requirements related to the function of comments in the dialogue contribute to qualifying a knowledge building process in collaborative learning.

Keywords: collaborative learning, knowledge building, dialogue, interaction, instruction, meta-communication, virtual environments

1. Introduction
Web-based learning is rapidly becoming a widespread alternative form of education (Crossman, 1997; Bates, 1999), not only within processes of continuous and lifelong learning, but also more and more as an alternative within traditional higher education (Harasim, 1999). The tangible advantages in terms of a more flexible, time and space independent organization of the learning process are clear. This learning potential of web-based environments has often been noted. The flexible organizational aspect appears to be an attractive feature, not only to lifelong and continuous learners, but also to the average campus student who wants to organize and manage his/her individual learning process the way it fits better with other aspects of his/her individual life conditions (Bates, 1999).
The flexible organizational design of web-based learning, by itself, does not make this method superior to other methods. Of more importance is the quality of the design of the learning process. There are many examples of web-based learning designs in which the web is utilized mainly as a "transmitter of information" and the learning process, as it were, limited to accessing and downloading information (Fjuk et al. 1999). Such designs offer no possibilities for interaction and collaboration among students and are rather sterile and lack a framework to stimulate collaborative knowledge building.

Designing for collaboration and interaction in learning is essential, "collaborative learning is predicated upon dialogue" (Harasim, 1989, pp 57). Establishment of effective dialogue and interaction has proven over the years to be a much more complex task than anticipated, despite the fact that "exchange of comments" seems to have been the essence of communication software systems such as Computer-Mediated Communication Systems (CMC) of the past and today’s communication software on the Web.

Considerable research has been focused on discovering within the design and /or delivery process key elements that foster collaboration. One suggestion is that the role and communicative behavior of the instructor is the key to the problem; other studies point to the nature of the collaborative activities (Harasim, 1999). It seems a well-established fact, however, that design elements relating to expected student responses play a major role in the amount and quality of interaction that occurs in a web-based CSCL-process (Collis, 1997).

But online interaction, even when it does occur, may not contribute to collaborative knowledge building. Basic research has addressed the quality of the interaction itself and reveals that much online interaction reflects the capabilities of the communication software (client-server systems as well as Web-based systems). Whereas support for "diverging" interaction through threading-facilities (Stahl, 1999) is readily available, software that facilitates "convergence" (e.g., fostering synthesis, generalization, etc.) is most rare.

We direct our attention in this paper toward the changes in responses of students that come about as a function of instructions and student-evaluation criteria relating to expected interaction and collaborative activities in a web-based course on Global Change (GC). We investigate the changes in quality of student interaction in three different deliveries of the course that emerge in response to three different types of instruction on expected behavior. From this we draw preliminary conclusions on the extent to which descriptions of expected behavior influence the interactive process and the quality of the interaction.

In section 2 we give a brief introduction of the web-based course producing the data for our analysis. Section 3 provides an account of our learning perspective and the criteria of quality used in the analysis of web-based dialogues. Section 4 addresses the nature and communicative conditions of web-based environments, forming part of the basis and rational behind our hypothesis. In section 5 we give a more detailed account of our research design and of the method used in our analysis, while section 6 provides the basic analysis. We reflect on the results of the analysis in section 7 and discuss future research perspectives.

2. Case
"Global Change" (GC), the web-based course which has produced the data of our analysis, is a conventional science course for senior undergraduates or beginning graduate students at a US university. It gradually has been migrated to a web base over the last 6 years, with new features being added as ancillary software has become available. Learner-centered activities in place of or supplemental to conventional lectures have been introduced.

Goals, content and collaborative design
The goals of the course are: (1) To help students come to an understanding of the interconnectedness of the global environment and the role of humans in charting (by design or default) its future trajectory, (2) to instill an appreciation for and recognition of authoritative literature on global-change issues, (3) to engage students within the course and across national and cultural boundaries in dialog on global-change issues, including ethical issues.

The content of the course addresses the human role in causing change of our global environment by having students address the scientific, societal, political and ethical issues surrounding such issues as climate change, ozone depletion, deforestation, desertification, biodiverisity, water degradation, and global human population (Takle et al, 2000).

The course consists of a sequence of learning modules on different global-change topics, each having evolved from a conventional university class time period. Each unit has a set of objectives, summary information on the topic, student-submitted collaborative (2-3 students) summary of class time discussion, "problems to ponder" as discussion starters for the electronic dialog, and extensive lists of web and other information on the learning module topic. Each unit has its own electronic dialog for student discussion among themselves and with outside experts or representatives of selected groups. Electronic dialog on individual learning unit topics is graded. The course is viewed by the designers as a laboratory for experimenting with a variety of pedagogical techniques and initiatives (Taber et al., 1997).

History of dialogue requirements:
The course went on the web in Spring 1995. Affiliated with the course was an opportunity to engage in electronic dialog. The use of dialog and associated instructor evaluation of student engagement in dialog changed over the years as follows:

1995: Electronic dialog was available separately for each lecture topic, and 5% of the evaluation of student performance was based on whether or not a student participated. There were no minimum requirements on amount of discussion and no attempt was made to judge quality as part of the evaluation of students.

1996: Same as 1995 except 15% of the grade depended on the electronic dialog. Some weak attempt was made to judge quality but primary criterion was participation and not quantity or content.

1997: Requirements for participation in electronic dialog were substantially enhanced. For full credit, the student was required to post at least 15 entries which had to include responses to at least 6 other students’ comments. Furthermore, these entries had to elicit comments from 3 other students. They also were required to respond to 3 questions issued by the instructor and to give their views on 3 additional ethical questions relating to global change. Thirty percent of evaluation of students was based on the these numbers but no evaluation of quality was attempted. A virtual portfolio was introduced for each student to manage discussion.

1998 and 1999: same as 1997.

2000: Same requirements as 1998 with addition of the knowledge-building (KB) process (Stahl, 1999) that required students to meet specific quality criteria in their discussion. Also, students were required to assess their own writing and demonstrate how they had met the criteria. Forty-two percent of the evaluation of student performance was based on discussion.

3. Criteria of quality in virtual communities of learning
To put our analysis of electronic dialogues in perspective, we briefly outline the set of criteria which we assert are signs of quality in a web-based collaborative learning process.

Learning is social
In the principles of collaborative learning, the process of learning is viewed to be a fundamentally social phenomenon, regardless of the varying theoretical emphasis in each single approach (Dillenbourg et al, 1995). Several other learning theories confirm this view, e.g. Etienne Wenger in his latest book, "Communities of Practice" (Wenger, 2000).

Awareness
The criterion of workspace awareness (Gutwin, 1995) allows students to take advantage of the opportunities for interaction that make collaboration a valuable way to learn. Gutwin et al. (1995) defines "workspace awareness" as the up-to-the-minute knowledge a student needs about other students' interactions with the shared workspace, as the most critical factor. The concept of awareness, as suggested by Gutwin may in this context be extended to also include an awareness at a meta-level in terms of the communicative function of a comment in a dialogue (see section 4).

Genuine collaboration
Salomon (1995) defines genuine collaboration as a result of genuine interdependence among people working together in a group, which requires supportive meta-structuring elements. More concretely, support of awareness and genuine collaboration in groups means providing structure for (Salomon, 1995):

• maintenance of overview of individual/collaborative learning expectations and progress;
• maintenance of interactions with peers and instructors;
• reflection, self-awareness, and feedback;
• evaluation throughout the learning process.

Creativity
Creativity is another criterion of quality in our learning perspective. Schmidt has a rather rational and compatible view on creativity (Schmidt, 1999) for the process of knowledge building. In his view, creativity is not a competency but rather a phenomenon which occurs in collaboration with others and which occurs in a space of confrontations between different interpretations (i.e. a knowledge building dialogue). Schmidt explains the process of creativity as departing from a situation of many perspectives, which gives more "weight" in the thought. This again creates more possible constellations of what one knows, which, in turn, elevates the opportunity for a new, original, creative thought or idea to emerge.

Learning takes place through collaborative knowledge building
The process of knowledge building in collaborative learning, first explored by Harasim (1989), involves mutual exploration of issues, mutual examination of arguments, agreements and disagreements, mutual questioning of positions, dynamic interaction and weaving of ideas (Harasim, 1989; Kaye, 1992; Sorensen, 1997). Mason (1993) finds this view to be in agreement with the communicative potential of the online environment, although she also points out the weaknesses of the online dialogue:

One of the complaints made about computer conferencing as an educational medium is that it does not lead to closure, to decisions, to final arguments or solutions. In fact, this is also its strength, for conferencing above all else is broadening, giving multiple perspectives and conveying the message that learning is a social, context-based construction, not the discovery of right answers or even a correct point of view.
(Mason 1993, p. 26)

In view of the generally recognized difficulties in fostering online student dialogue that converges (e.g., synthesizing) rather than diverges (noted by Mason), Stahl (1999) suggested a set of factors that characterize quality in the knowledge building process:

• BRAINSTORMING is the introducing of new ideas that relate to the topic or task and offer a perspective not previously considered.
• ARTICULATING includes explaining complex or difficult concepts.
• REACTING provides an alternative or amplified perspective on a concept previously introduced by a student
• ORGANIZING refers to assembling existing thoughts or perspectives in such a way that a new perspective emerges.
• ANALYSIS includes comparing or contrasting previously articulated views or puts new understanding on existing data.
• GENERALIZATION takes comments or data already presented and extracts new information or knowledge that applies to a broader set of conditions.

Implementing these learning quality criteria of collaborative knowledge building requires a corresponding meta-functional pedagogy or instruction which facilitate and motivates such collaborative dialogue (see section 4).

4. Dialogues in distributed CSCL: Reflection primary to involvement
Promoting knowledge-building dialog within the context of collaborative learning is complex. Considerable research devoted to identifying causes of this problem has been directed towards the role and communicative behavior of the instructor. Some of the important findings were that at the same time as being occupied with the style of discourse in the learning process, the tutor’s main role and investment should be directed towards the art of "weaving" (Feenberg, 1989) in the language games of the interactive process of learning:

Meta-comments concerning the content of the discussion are called "weaving" comments. They summarize the state of the discussion, identifying its unifying themes and points of disagreement. These comments reveal an important benefit of textual mediation for social interaction. Writing a weaving comment involves a relation to discourse which is characteristically literary and encourages a command of the written world (...). Weaving comments allow on-line groups to achieve a sense of accomplishment and direction. They supply the group with a code for framing its history and establish a common boundary between past, present and future.
(Feenberg 1989, pp. 34-35)

Other research has focused on the design and nature of the collaborative activities implemented in the learning process (Collis, 1996).

Both of these perspectives assume that the main causes of the problem concerning interaction and collaboration lies in the instructional and pedagogical aspects of the learning process. There is no doubt that both of these research perspectives address very pertinent aspects of virtual learning designs and deliveries, but it is quite unlikely that they alone constitute the whole story.

Alternative studies examined the quality of the virtual environment and the evolvement of electronic inter-human dialogue (Scardamalia & Bereiter, 1996), which directs our attention toward the nature of the online environment as a facilitator of interaction in learning. Some of these studies have been directed toward clarifying the human dialogue and the new conditions of its practice (Sorensen, 1993), and are based on the hypothesis that the nature of human dialogue changes according to the its new, asynchronous conditions in the virtual environment. Therefore, the way the dialogue is established and practiced changes accordingly. Significant findings from these studies indicate, that the social aspects of interhuman dialogues suffers in electronic dialogues under the asynchronous written conditions, and, therefore, electronic dialogues need to invent new ways of bringing the social aspects of the relationship with the group into being. The rational behind this — and the important aspect from the perspective of collaborative learning - is that the social aspect of a group process is an essential motivator for collaboration in online learning (Harasim, 1993; Cornell & Martin, 1997), in the sense that it seems to carry a rather high proportion of a group member’s inclination to engage in any interaction with the group at all (Moore & Kearsley, 1996). This seems in total agreement with the widely acknowledged insight, that inclination to interact online is sensitive to the perception of interaction (Gunawardena, 1995).

Similar research has focused on the nature and appropriateness of the communicative scaffolding in electronic dialogues (Sorensen, 1999). This approach is rooted in a perspective on the electronic environment as being fundamentally altered with respect to its "ontological" conditions for (co)existence. The perspective views the move from physical reality to a distributed virtual symbolic environment as being so radical, that it causes a change of principles and premises of being and acting communicatively. In other words, a new dialogical paradigm. The rational behind this view is that the online universe is a meta-communicative world. Contrary to the physical world in which involvement is viewed to be primary to reflection (Heidegger, 1986) - the virtual universe provides a context and an "ontology" in which reflection may be said to be primary to involvement (Sorensen, 1999).
Assuming this new dialogical paradigm, and this primary position of reflection and meta-communication, it is very conceivable, from the perspective of collaborative interaction and dialogue in CSCL, that also the task of scaffolding learning processes which aim at supporting both interaction and (self)reflection, must move at a meta-level in terms of creating awareness of the function in a dialogue of a contributed comment.

This position gains support from studies concerned with the role and function of the online tutor. Several of these point to the role as being primarily concerned with facilitation of meta-communicative acts and meta-interactive processes. Andrew Feenberg (1989) introduced the term "weaving" to cover such processes of meta-communication, like e.g. contextualizing (open discussion, frame of discussion, agenda, etc.), monitoring process (react on comments, misunderstandings, etc.), and meta-commenting (problems in context, norms, agenda, etc.; synthesizing) (Feenberg, 1989).

5. Research design
In our analysis we wanted to investigate the role and nature of the instructions in the requirements given to the students, stimulating them to interact. We wanted to assess whether meta-communication in terms of providing meta-awareness of "the function of a comment" in the knowledge building process, will improve the quality of the knowledge building dialogue, according to the definition of quality as suggested by Stahl (1999).

In our research design and analysis we make the following assumptions:

1. Characteristics of discussion and comments that contribute to a knowledge-building process (KB) are (Stahl, 1999): brainstorming, articulation, reaction, organization (including synthesis), analysis, and generalization.

2. Use by students of characteristics of the KB-process in written dialogue contributes to student learning (as characterized in section 3).

Our hypothesis for the analysis is then, that by explaining the characteristics of a KB-process, and by grading (Sorensen & Takle, 1999) student discussion on the basis of their reflected use of these characteristics, student will measurably increase their use of these characteristics. The rational behind the hypothesis is an acknowledgement of the need in primarily reflective virtual environments (as described in section 4) for communicatively providing meta-awareness in relation to expected communicative actions (as described in section 3).

We have evaluated characteristics of student dialogue and its relationship to course requirements by assembling 10 comments from 1995, 10 from 1997, and 10 from 2000. These were drawn at random (although not strictly with statistical rigor) and represent comments made under quite different criteria for evaluation of student performance.
In drawing the comments from the respective databases, care was taken to draw comments from the same (or very closely related) topics in each of the three years.

The data sets represent (a) comments with no obligations attached as far as student performance evaluation (1995), (b) comments when numbers of responses and numbers of interactions among students were used for evaluation (1997), and (c) comments when the KB requirements were imposed. For 2000, we also have, for some database entries, the student self-assessment of their own writing. We also have assigned a numerical value from 1 to 10 representing the quality of the comment.

6. Analysis of dialogues
For the purpose of our analysis we have categorized the analyzed comments in the online dialogues according to the knowledge building quality requirements presented by Stahl (1999) in section 3:
comment with no KB characteristics

• question (a request for clarifying information)
• analysis (comparing or contrasting previously articulated views)
• articulation (the explanation of complex or difficult concepts)
• reaction (presenting an alternative or amplified perspective on a concept previously introduced)
• brainstorming (the introduction of new ideas that relate to the topic or task and offer a new perspective)
• organization (the organization of existing thoughts or perspectives)
• generalization (extracting new information or knowledge from comments already presented, that applies to a broader set of conditions)

A level of quality (1-10) was subjectively assigned to each comment. The mean quality went from 4.4 to 3.2 to 5.3 from 1995 to 1997 to 2000, respectively. In 1995 a preponderance entries were questions, whereas in 1997 there were more reactions to other students. In 2000, more student comments were characterized as articulation.

From these results, we draw the following tentative conclusions. In 1995, with no requirements on volume or quality of writing, students used the electronic dialog for asking questions. The questions did not seem "forced" since their postings had essentially no bearing on how they were evaluated in the course.

For 1997, the overall quality of the postings went down. There seemed to be more "forced" responses. More questions were of a rhetorical nature, rather than seeming to originate out of curiosity about the subject matter. We attribute this to a sense of urgency on the part of students to meet the volume obligations of the evaluation scheme.
The increase in quality from 1997 to 2000 suggests that students responded to the demands for higher quality of writing by meeting the requirements of the KB process. A notable difference in the 2000 discussion as compared to previous years was the reduction in the "social" or "chat" nature of the comments. But at the same time, the number of comments increased. Comments were much more like individual essays and less spontaneous.

Perhaps the largest difference over the change in dialog requirements was in the length of each entry. In 1995 the average length of the 10 entries was 88 words and in 1997 the average was 93 words. In 2000, however, the average soared to over 2,500 words. And the quality of the grammar of each posting increased markedly as well. This likely is attributable to the fact that students wrote their postings as essays in word processors on their own computers, subjected them to spell checking, and reviewed the grammar before cutting and pasting them into the course web-site. This increased quality and quantity came at the expense of spontaneity and "social" content.

7. Discussion and future perspectives
How do we evaluate the results of our analysis? - Assuming that meta-awareness of KB-characteristics among students as well as grading of comments increase the process of collaborative learning, and accepting the value of the knowledge building characteristics used in our analysis, we think it is fair to say that our little experiment have proved itself to be relevant. Over the three trials the quality of student comments has increased, both as a result of grading requirements and as an effect from the meta-awareness around the functions of the requested comments.

Our sample size was quite small and therefore the results are subject to large uncertainty. Therefore, the extent to which the results can be generalized may be questioned. The selected comments, however, were drawn at random and taken out of the social context of the specific dialogue in which it appeared. Perhaps this could be considered a problem of the data set, since the knowledge building requirements are concerned with the sequential development of online interaction.

As mentioned (section 6), the largest difference over the change in dialog requirements was in the length of each comment, and the increased quality of the grammar used. It seems that the more carefully composed essay-like style of comments as well as the increased quality and quantity came at the expense of spontaneity and "social" elements in the interaction. It is conceivable that dealing with electronic dialogues and an electronic comment in terms of a request for reflection on its function, is inhibiting the evolvement and practice of a spontaneous and dynamic dialogue. Back in the 80’ties a similar conflict was demonstrated with the design of "The Coordinator" (Winograd & Flores, 1986), a communication system designed on a fundamental principle of reflection on the function of every planned communicative act. Among other things, this design did not allow for the growth of the dynamic and social elements in an interaction.

As such, the question could be posed, whether the strong meta-reflective requirements enforced on a dynamic phenomenon as human interaction (even though it is not in terms of formalization) appears problematic. Or perhaps whether the lack of recognition of social elements in the interaction works against an incitement to engage in collaborative interaction or knowledge building with others. Moreover, it may be pertinent as a basis for a potential conclusion around this aspect to specify, whether one is using data from an off-campus course (where the students have no complementary social interaction) or an on-campus course (where social interaction among students is part of daily life) as in the present study.

In the data from our experiment there is some indication, that the "forced requirements" on the collaborative dialogue actually functioned like a "kick-off" to engage in dialogue, and that it actually caused more interactivity than required to appear. But we find the data set from this study too small to conclude more precisely around this aspect. It is beyond any doubt, though, that - as default - to be forced to reflect on the function of one's action before carrying it out is likely to decrease spontaneity, phatic behavior and high interactivity. Above all, we believe that the optimistic results of our analysis have demonstrated that, once again, the problem is a complex one. And perhaps the usual controversial points to discuss in this respect will be "how" to do it and "to what extent". All these are questions to be further studied and pursued in the future.

Acknowledgments
We acknowledge the contribution of Kathryn St. Croix, Iowa State University, for collaborating with us in the analysis of dialogues. Contributors to the design and implementation of the Global Change course include Doug Fils, Michael Taber, Jennifer Hodson, and David Flory. The course is implemented under the auspices of the International Institute of Theoretical and Applied Physics.

References
Bates, A. W. (1999): "Managing Technological Change: Strategies for Academic Leaders" San Francisco. Jossey Bass. 1999.

Collis, B. (1996): Tele-learning in a Digital World. International Thomson Computer Press. 1996.

Collis, B. (1997): Experiences with Web-based environments for collaborative learning and the relationship of these experiences to HCI research. Published in the Proceedings of the Working Confernce of IFIP WG 3.3, Sozopol, Bulgaria.

Cornell, R. & Martin, B. L. (1997): The Role of Motivation in Web-Based Instruction. In Khan, B. H. (ed.): Web-Based Instruction. Educational Technology Publications Englewood Cliffs, NJ. 1997. pp. 93-100

Dillenbourg, P., Baker, M., Blaye, A., O’Malley, C. (1995) The Evolution of Research on Collaborative Learning. In Reimann, P., Spada, H. (eds.) Learning in human and machines. Towards an interdisciplinary learning science, London: Pergamon, (pp. 189-211)

Feenberg, A. (1989): The Written World. In Mason, R. & Kaye, A. R. (eds.): Mindweave. Communication, Computers, and Distance Education. Pergamon Press. Oxford. 1989. (22-40).

Fjuk, A., Sorensen, E. K., Wasson, B. (1999): Incorporating Collaborative Learning, Networked Computers and Organisational Issues into Theoretical Frameworks. In: The New Educational Frontier: Teaching and Learning in a Networked World: Proceedings of ICDE'99. 1999.

Gunawardena, C. N. (1995): Social presence theory and implications for interaction and collaborative learning in computer conferences. International Journal of Educational Telecommunications, 1(2/3), pp. 147-166.

Gutwin, C., Stark, G., Greenberg S. (1995): Support for Workspace Awareness in Educational Groupware. The Proceedings of CSCL95, available from Lawrence Erlbaum Associates, Mahwah, NJ http://www-cscl95.indiana.edu/cscl95/ .

Harasim, L. M. (1989): Online Education. A new domain. In Mason, R. & Kaye, A. R, (eds.): Mindweave. Communication, Computers, and Distance Education. Pergamon Press. Oxford. 1989. (50-62).

Harasim, L. (1999): The Virtual-U Field Trials: Lessons about Teaching and Learning Online. 1999.

Harasim, L. (1993): Collaborating in Cyberspace: Using computer conferences as a group learning environments. Interactive Learning Environments, 3(2), pp. 119-130.

Heidegger, M. (1986): Sein und Zeit. Max Niemeyer Verlag Tübingen. 1986.

Kaye, A. R. (1992): Learning Together Apart. In Kaye, A. R. (ed.): Collaborative Learning Through Computer Conferencing. NATO ASI Series, vol. 90. Springer-Verlag, 1992. (pp. 1-24)

Mason, R. D. (ed.) (1993): Computer Conferencing. The Last Word. Victoria. Beach Holme Publishers.

Moore, M. G. & Kearsley, G. (1996): Distance Education. A Systems View. Belmont, CA: Wadsworth.

Salomon, G. (1995): What Does the Design of Effective CSCL Require and How Do We Study Its Effects? In
Schnase, J. L., Cunnius, E. L. (eds.): CSCL 95. Computer Support for Collaborative Learning, pp. 147-156. Lawrence Erlbaum Associates Inc., Mahwah, NJ.

Scardamalia, M. & Bereiter, C. (1996): Computer support for knowledge building communities. In Koschmann, T. (ed.): CSCL: Theory and Practice of an Emerging Paradigm. Lawrence Erlbaum Associates, Hillsdale, NJ, pp. 249-268

Schmidt, H. H. (1999): Diagnosis I-II-III. Danmarks Pædagogiske Institut. Copenhagen. 1999.

Sorensen, E. K. (1991): Dialogues in Networks. In Andersen et al. (eds.): The Computer as a Medium. Cambridge University Press. Cambridge 1993. (pp 389-421).

Sorensen, E. K. (1999): Intellectual Amplification through Reflection and Didactic Change in Distributed Collaborative Learning. In C. M. Hoadley and J. Roschelle (Eds.), Proceedings of the Computer Support for Collaborative Learning (CSCL) 1999 Conference (pp. 582-589). Palo Alto, CA: Stanford University [Available from Lawrence Erlbaum Associates, Mahwah, NJ]. http://kn.cilt.org/cscl99/A71/A71.HTM

Sorensen. E. K. (1997): Learning in Virtual Contexts. Navigation, Interaction, and Collaboration. Ph.d.-afhandling. Aalborg University. Danmark. 1997

Sorensen, E. K. & Takle, E. S. (1999): Distributed Collaborative Learning across Disciplines and National Borders. In C. M. Hoadley and J. Roschelle (Eds.), Proceedings of the Computer Support for Collaborative Learning (CSCL) 1999 Conference (pp. 575-581). Palo Alto, CA: Stanford University [Available from Lawrence Erlbaum Associates, Mahwah, NJ]. http://kn.cilt.org/cscl99/A70/A70.HTM

Stahl, G. (1999): Reflections on WebGuide: Seven Issues for the Next Generation of Collaborative Knowledge-Building Environments. In C. M. Hoadley and J. Roschelle (Eds.), Proceedings of the Computer Support for Collaborative Learning (CSCL) 1999 Conference (pp. 600-610). Palo Alto, CA: Stanford University. [Available from Lawrence Erlbaum Associates, Mahwah, NJ]. http://kn.cilt.org/cscl99/A73/A73.HTM

Taber, M. R., E. S. Takle, and D. Fils (1997): Use of the internet for student self-managed learning. Preprints, Sixth Symposium on Education. American Meteorological Society, 2-7 February, Long Beach, CA.

Takle, E. S., Taber, M. R., and Fils, D. (2000): An interdisciplinary internet course on Global Change for present and future decision-makers. In A. Van der Beken, M. Mihailescu, P. Hubert, and J. Bogardi, 2000: Proceedings of the Learning Society and the Water Environment. European Commission Press, Luxembourg. pp. 264-268.

Wenger, E. (2000): Communities of Practice. Learning, Meaning, and Identity. Cambridge University Press. 2000.

Winograd, T. & Flores, F. (1986): Understanding Computers and Cognition. A New Foundation for Design. Addison-Wesley. 1986.