Division of Research School of Business Administration THE IMPACT OF FACE-TO-FACE COLLABORATIVE TECHNOLOGY ON GROUP WRITING Working Paper #633 Marjorie Horton, Electronic Data Systems Priscilla Rogers, University of Michigan David Brimm, University of Michigan Laurel Austin, Camegie Mellon University Michael McCormick, Electronic Data Systems FOR DISCUSSION PURPOSES ONLY None of this material is to be quoted or reproduced without the expressed permission of the Division of Research Copyright 1990 University of Michigan School of Business Administration Ann Arbor, MI 48109-1234 April 1990

I i i i q i i I I i I I i i I I i I I I I I

THE IMPACT OF FACE-TO-FACE COLLABORATIVE TECHNOLOGY ON GROUP WRITING Marjorie Horton1, Priscilla Rogers2, David Brimm2, Laurel Austin3 & Michael McCormick1 1Center for Machine Intelligence Electronic Data Systems 2001 Commonwealth Blvd. Ann Arbor, Michigan 48105 2University of Michigan School of Business Administration Ann Arbor, Michigan 48109 3Carnegie-Mellon University Department of Social and Decision Sciences Pittsburgh, Pennsylvania 15213 - I

I i I I i I II I i I I i I i

The Impact of Face-to-Face Collaborative Technology on Group Writing Marjorie Horton Center for Machine Intelligence Electronic Data Systems 2001 Commonwealth Blvd. Ann Arbor, Michigan 48105 Laurel Austin Carnegie-Mellon University Department of Social and Decision Sciences Pittsburgh, Pennsylvania 15213 Priscilla Rogers and David Brimm University of Michigan School of Business Administration Ann Arbor, Michigan 48109 Michael McCormick Center for Machine Intelligence Electronic Data Systems 2001 Commonwealth Blvd. Ann Arbor, Michigan 48105 ABSTRACT This experimental study examines the impact of face-to-face collaborative technology on group writing. We conducted a comparative analysis of small work groups writing managerial memoranda with the collaborative technology of the Capture Lab and with conventional writing tools. We find that the technology significantly alters the writing process, resulting in less initial group planning, more individual work and more revising, compared to when conventional tools are used. The minimal group planning in the technology condition appears to impact the group processes of negotiation and consensus-reaching. The technology also influences group interaction patterns, resulting in more individual tool use, less speech and less group focus. User feedback suggests the technology can have both positive and negative effects on communication among group members. We found no general effects of the technology on document quality, although preliminary evidence suggests that writing with technology can enhance writers' audience adaptiveness. We also present some comparisons of how different groups used the technology and how these patterns may have influenced document quality. The final sections discuss suggestions for effective use of collaborative technology and future research. 1

r I -1. i I I

INTRODUCTION Despite the increasing number of experimental computer-supported meeting rooms for face-toface group work, we still have much to learn about the impact of these tools on collaboration. How can groups actually use this kind of collaborative technology to accomplish complex tasks and how do these tools influence the nature and quality of their work? Much of the pioneering work on computer-supported meeting rooms focused on providing specialized groupware tools to support particular types of work in meetings, such as Colab's Cognoter for planning presentations (Stefik, Foster, Bobrow, Kahn, Lanning & Suchman, 1987), Project Nick's communication tools for software design teams (Rein & Ellis, 1988), and Minnesota's SAMM system for structured problem solving (DeSanctis & Dickson, 1987). In our research we are focusing instead on how groups collaborate using a computer-supported meeting room (the Capture Lab) designed for broad functionality, in which users can work with a shared computer, at private workstations, and use whatever conventional software they find most suitable for their specific tasks (Mantei, 1988). For this initial study, we chose collaborative writing as the task domain, for several reasons. Collaborative writing is a rich domain to study because it is a complex and potentially difficult task, encompassing many activities inherent in group work -- planning, problem-solving, negotiation and decision-making. Recent empirical surveys of writing in the workplace reveal that collaborative writing occurs quite frequently and that in some professions, it is more frequent than individual writing (Couture & Rymer, 1989). Composition and communication scholars also acknowledge the need to study face-to-face collaboration and the impact of technology (Forman & Katsky, 1986; Forman, forthcoming). Another reason for our choice of tasks is that observations of Capture Lab users over the past two years suggest that this technology is particularly beneficial for groups working on written documents (Elwart-Keys & Horton, in press). Finally, we felt we could objectively assess the quality and the efficiency of collaborative writing more easily than some other kinds of collaborative tasks (e.g., strategic planning, product reviews, etc.). The focus of this study was, how does face-to-face collaborative technology impact group work, in particular collaborative writing? To address this question, we completed a comparative analysis of groups writing managerial memoranda with the computer technology and with conventional writing tools and evaluated both the writing process and the written product. 2

In the following sections we describe our research methodology and data collection. We then discuss our research findings in three sections focusing on the impact of the technology on the writing process, on group interaction, and on document quality. The fourth section explores patterns of group work and tool use as they relate to the quality of a group's work. In conclusion we offer suggestions for how groups can use this kind of technology for effective collaboration and we identify questions for further research. METHODOLOGY Experimental Design To control for possible group effects, we used a within-groups design, comparing two experimental conditions for collaborative writing. All groups produced one document using conventional writing tools (the Non-technology (NT) condition) and one document using the collaborative technology of the Capture Lab (the Technology (T) condition). Groups produced both documents in the same meeting room. For this initial study we purposely kept these two conditions distinct. In the T condition we did not allow subjects to use traditional writing tools such as pens, paper and flipcharts; in the NT condition we did not allow subjects to use any computer tools. Non-technology (NT) condition Each group member had a pad of paper, a pencil, and a pen. A flipchart on an easel, markers and tape were also provided. A clock was mounted on the wall. Technology (T) condition Each group member had a private computer (Macintosh IIP") consisting of a display monitor, keyboard and mouse. This computer equipment was positioned around a large rectangular conference table.1 Group members also shared a public MacintoshTM. Its output was projected on a large monitor at the front of the room. There was no keyboard or mouse for this computer. All of 10ur subject groups used the Capture Lab technology moved to a conventional meeting room, since the Capture Lab was under reconstruction. Since this was a temporary meeting room during the reconstruction, the monitors were not embedded into the conference table as in the Capture Lab. 3 ------------- --- -— I~-~-~-~ --- -I- lccc-"cc~ — lp --- —--— ~-~ ~;

the central processing units were stored remotely. Additionally, members each had access to a laser printer which was in the room. See Figure 1 for the room layout.2 Printer J Y.....: O Public Screen Figure 1 Diagram of Meeting Room All members of the group also had equal access to the public computer. To access the shared computer, a user pressed a special key on his or her keyboard. Pressing this key again returned the user to his or her individual computer. Only one user could control the public computer at any one time. If one user had control of the public computer and another user pressed the access key, the first user was reconnected to his or her personal computer and the second user gained control of the public computer. A clipboard transfer function allowed users to move sections of a document between their individual computers and the public computer using the Macintosh cut and paste functions. All computers also displayed a small clock in the menu bar. MS WordT was open on the public and private computers at the start of the session. For this initial study we deliberately decided against providing subjects with other software tools for the task, such as planning or brainstorming applications. We wanted to avoid giving them too many tools to learn or setting up any kind of task demand for planning or brainstorming. Subjects Five groups of advanced business and liberal arts undergraduates participated in this initial study. Three of these groups, from a business writing course, participated as a course requirement. Due to unexpected low enrollment in this course, we recruited students from another business communications course and paid them for their participation. Three groups included four members, one included five and one included three. 2A fifth computer was added for the one group comprised of five members. 4

All of the groups had some prior experience working as a group. The extent of this experience ranged widely, from a few class assignments for several of the groups, to several lengthy class projects over the course of one year for the most established group. Four of the five groups could be considered relatively immature, in terms of group members' knowledge about the abilities and values of the other members (Jewell & Reitz, 1981). Writing tasks We used two comparable writing tasks, counterbalancing order of task, order of condition and type of group (paid vs. course requirement). Both the tasks and the experimental procedures were piloted with student and professional groups. Both tasks were short cases that involved groups in politically-sensitive business situations that compelled them to write a persuasive response in memorandum form. These tasks were developed to model real business situations that would warrant a group writing effort. Each case included information on the group's role, the motivation for the memorandum, and information about who would receive it. (Appendix 1 includes both tasks). For one task, group members were assigned the role of a sales management team, writing to an executive committee to argue against proposed changes in a vital sales training program. For the other task, group members were assigned the role of regional directors, writing to store managers to dispel rumors about changes in hiring practices. Groups were given up to two hours to complete each task and were instructed to leave all materials they produced during that period, including software files, printouts, brainstorming lists, drafts, and finished documents. (The groups that were paid for their participation were told they would be paid for the entire session even if they finished early.) Training Observational research on the Capture Lab has shown that lack of expertise with the technology is an impediment for users in computer-supported meeting environments (Elwart-Keys, Halonen, Horton, Kass & Scott, 1990). To minimize such effects, all groups participated in hour long training sessions on the MacintoshTm and MS WordT. For many subjects, this training was a review since they had worked with the MacintoshT and MS WordT previously. All subjects were computer literate before beginning this project. 5 ~ --- — I~^-C-l-"`~I~ L~-~~~-~-~~~3~I

Training specific to the collaborative technology was also provided at the start of each group's technology session, and subjects had an opportunity to practice prior to beginning the task. This training included a few general comments about possible ways groups could use the public and private computers (e.g., using multiple windows, taking notes or developing ideas on one's private machine). For purposes of control, in the NT condition the experimenter conducting the training also commented about ways groups could use the conventional tools (e.g., taping flipchart pages on the wall, taking notes on individual notepads). Despite this preliminary training, we acknowledge that subjects began their participation in the study at different points on the learning curve for the T versus NT conditions, since they all had years of experience writing with conventional tools and no prior experience using this kind of groupware. Not providing more extensive training allowed us to explore the creative strategies and patterns subjects developed on their own. Clearly any conclusions we can draw about the effectiveness of the technology should be considered tentative in light of the subjects' limited experience with it. Data Collection A video camera positioned in one corner of the room recorded the sessions and transmitted a video image of the session to a remote observation station. In the T condition, the output of the public screen was displayed on a computer at the observation station via Timbuktu~ software. Additionally, all of the subjects' screen activities on both the public and private computers were recorded with ScreenRecorder~, a program which creates a software "tape" of computer screen activity and which can be played back at a later time. Real-time logging of group activities Each session was observed by two or three researchers. Overall, there were four observers during the course of this study. During a particular session, at least two researchers alternated logging the group's activities for half-hour periods. To log group activities observers used a 6 N

specialized software logging tool (GroupCoder) developed at the Center for Machine Intelligence. Figure 2 shows the main logging screen for this tool. I ~ I ObserverEperim.enter.. Group T3 Date 2/12/90,-r~~~~spech (Load Protocol) speech group-FOCUS New Session | erge Protocol [~lion~ ~ ~planning PUBLIC tooJ producing private.tool (Create sta StopClock Done Stkcard o00:00:3 001. 3 _ l --- l Help nontask oops lReset I Reset).Beep on Save Protocol Comment s: to r - - - I Figure 2 GroupCoder Screen Logging involved watching the group from the observation station and recording the collaborative activities. Every thirty-seconds a tone would sound and the researcher would log any of eight activities occurring during that interval by selecting the appropriate code buttons. Additional comments or annotations could be recorded in the comment field. To increase the accuracy of logging, the researcher could also select an "oops" button to indicate a logging error, comment on the nature of the error, and later revise the logging protocol accordingly. The eight logging categories are described in Table 1, organized into three main groups —writing activity, group interaction, and tool use. 7

Writing Activity Planning: discussing the Producing: handwriting, Revising: handwriting, writing situation or ideas for typing, or pasting to create text typing, pasting, erasing or document organization, content for the group memorandum. deleting to change existing text or style; generating plans such for the group memorandum. as an outline or list of brainstorming ideas. * Planning, producing, and revising were logged when those activities occurred on public tools. Group Interaction_____ Speech: at least one person Group focus: all group Non-task: extended (more spoke during the 30-second members were involved in the than several seconds) activity interval. same task-oriented activity for by an individual or by the the majority of the 30-second group that is not related to the interval. The subjects needed task. to be attending to the same aspect of the same task, not working independently.____ Tool Use Private tool use: Public tool use: T condition: using an individual computer(s) T condition: using the public computer (including transferring control and moving the (including transferring control and moving the mouse). mouse). NT condition: using an individual note pad(s). NT condition: using the flipchart or group An individual notepad is any notepad used for notepad. A group notepad is any notepad used private work such as notes, private production of for recording text for the group memorandum. sentences, etc. This is in contrast to the individual notepad described to the left. Table 1 Logging Definitions The logging categories were refined over several pilot sessions. These pilots allowed us to discuss observations, resolve disagreements, and revise the logging categories. We have noted consistency between the results from analyses of the logging data and our findings from the videotape scenarios, described below. Currently we are assessing inter-rater reliability for the logging scheme. Questionnaires Prior to their participation in the first session, subjects completed a brief profile that included questions about their proficiency with computers, typing and word processing software, and their attitudes towards writing. (Analysis of these profiles revealed the groups were comparable in terms of proficiency, but varied somewhat in their attitudes towards writing). A post-session questionnaire using both open-ended and multiple choice questions on a 5-point scale focused on the subjects' evaluation of the writing tools, group dynamics, and work satisfaction. 8

Videotape scenarios The videotapes from all of the experimental sessions were reviewed independently by three researchers, who individually wrote scenarios describing each group's writing process, patterns of tool use, and group dynamics. Comparisons of these scenarios indicate a high degree of consistency between the researchers, which was confirmed by extensive follow-up discussions about the sessions. Document Assessment Writing consultants from the Michigan Business School Managerial Writing Program, experienced in assessing managerial writing, used a number of holistic and analytic measures to evaluate the quality of the final documents. These measures were developed to evaluate managerial documents. Holistic measures included a general seven-point holistic assessment, a within-group comparative evaluation (Haas, 1989) and a scale ranking the documents from best to worst. The analytic measures used were: 1) the Persuasive Appeals Measure, 2) the Persuasive Adaptiveness Measure, 3) the Tone Measure and 4) the Quality of Reasoning Measure (Rogers, 1989). To insure that these evaluators were blind to condition, all the memoranda were retyped in the same font. FINDINGS Researchers in the emerging field of computer-supported cooperative work are still learning what kinds of tools and what kinds of data are most valuable in building our understanding of collaborative work. In our efforts to understand how groups can use the Capture Lab technology for collaborative work, we have collected a wide variety of data. For this paper we pulled together a subset of findings we believe tells the most informative and interesting story about the impact of the technology on group writing. The findings below are drawn from the logging data3, a comparative analysis of the videotape scenarios, a review of all of the groups' work products (including planning notes, etc.), the questionnaires and the document assessment. We discovered that technology significantly alters the group writing process and more generally, that it impacts the nature of group interaction. However, it may only slightly change the quality 3For the logging data reported in this paper we found no overall order effects, so order of condition will not be discussed further. 9 -------------------------------— ^ " 1* ^`~-~~1-~11 ---~-~ —~~II. ---11~1 C-~-I(-~~ 7~ --- —-CI

of a group's final product. While we focused on collaborative writing, we believe these preliminary findings have implications for how groups work on many different problem-solving tasks using face-to-face collaborative technology. Technology Alters the Writing Process Our most significant discovery may be that groups used substantially different patterned processes to complete the writing tasks in the T and NT conditions. Generally, the writing process has been said to involve several distinct activities, including initial planning, composing and revising (Hayes & Flower, 1980; Smith & Lansman, 1989). Groups engaged in each of these activities to some degree in both conditions. However, the amount of time spent on some of these writing activities, the amount of group versus individual work and the patterns of tool use varied dramatically according to condition. In presenting our findings, we examine groups' planning, producing and revising, in turn; however, we should stress that these are not necessarily linear stages of writing, but rather are often iterative processes. (In discussing data for individual groups, we will refer to Groups 1-5 as NT1 - NT5 and T1-T5 to relate the group IDs to condition.) Less Initial Group Planning With Technology Given our broad definition of planning (see Table 1), it's not surprising that groups typically planned throughout the writing process. However, we focus here on initial planning -- planning prior to text generation —because it was both the extent and the nature of initial planning that varied considerably between conditions. (By our definition, initial planning does not include time group members spend determining procedures for working on the task, e.g., whether to divide up the work, what writing tools to use, etc.) We found that the NT condition prompted initial group planning, in contrast to the T condition, in which group members typically planned individually, if at all, and often without subsequently sharing their plans. (By our definition, group planning involved group members working together to plan their document. Individual planning involved group members working alone, generating individual plans and notes.) Table 2 summarizes these results. 10

Non-technolog ___ _Technology Individual Group Individual Group Group ID Planning Planning Group ID Planning Planning 1 0 12 1 0 4 2 0 30 2 3 1 3 0 25 3 0 0 4 0 27 4 10 11 5 0 20 5 12 23 Avege 0 22.8 Average5.0 7.8 Table 2 Initial Planning Times (time in minutes) In the NT condition, all groups engaged in initial planning as a group, for an average of 22.8 minutes. Groups generated brainstorming lists and outlines —three groups used the public flipchart extensively; in another group, members took private notes. Only one group failed to write down any of its group planning work. We saw no individual initial planning. By contrast, in the T condition, most groups usually began producing text with little or no initial group planning; across the five groups, the average time spent was 7.8 minutes. This difference in amount of initial group planning across conditions was marginally significant (paired-t = 3.6, p =.06 (two-tailed)). As Table 2 shows, in the T condition one group spent no time, while two other groups only planned as a group for up to four minutes. One of these groups (T2) spent just one minute (and only discussed the memo header). Another group (T4) spent a total of 11 minutes planning as a group, but this time was interspersed with periods of individual work in silence. Four of the five groups did not use the public computer to capture any of their group plans in writing. The fifth group (T5) was a clear exception to this general pattern. Members of T5 spent 23 minutes on initial group planning, but only after a considerable period of individual planning. In the T condition, members from three of the groups spent time on initial planning as individuals, using their private computers to generate brainstorming lists and outlines. Across the five groups, the average time spent on initial planning as individuals was 5.0 minutes. However, only one group (T5) spent time explicitly reviewing and merging the work from its period of individual planning. Questionnaire data indicates that subjects were discontent both with the quality of the planning and the functionality of the writing tools for planning in the T condition. Subjects rated the quality of their planning in the T condition lower than the quality of their planning in the NT condition. 1 1

When asked to evaluate aspects of their group's writing process, individual subjects rated planning as only "poor" to "good" after the T session (mean = 2.90), but as "good" to "very good" after the NT session (mean = 3.35), (paired-t = -2.95, p <.02, two-tailed). Moreover, after participating in both conditions subjects were asked which set of writing tools they would prefer to use for planning group documents. A surprising 75 percent selected the conventional writing tools. There are a number of possible reasons for this response. Although our subjects were computer literate, none had experience with collaborative computer technology before their participation in the study. This lack of familiarity may have inhibited their computer use for planning. Moreover, the fact that we did not provide subjects with any computer tools specifically designed for brainstorming and outlining may have been a factor. However, subjects also reported that when they write individual papers on computers they typically plan on paper before writing. We do not argue that less time planning and less group planning per se are detrimental to a group's ability to write collaboratively. In fact, individual work can be more effective and result in higher quality results than group work, as research on brainstorming has shown (Diehl & Stroebe, 1987; Vroom, Grant, & Cotton, 1969). However, we hypothesize that some of the difficulties -groups had drafting in the T condition, as we will discuss, stemmed from their lack of planning and consensus-reaching early in the writing process. More Individual Producing With Technology We also find quite different patterns of group versus individual work in how groups produce their memoranda. In the NT condition, four of the five groups actually composed primarily as a group. Typically one or two members of the group served as the scribe(s), recording text on the "group notepad" as it was generated verbally by the group.4 (One of these groups used the flipchart as the "group notepad" for much of their session.) Across groups, individual efforts were relatively minor in the NT condition —individuals occasionally jotted down ideas for future reference. We saw only two salient examples of extended individual effort in the NT condition. In one case (NT2) one group member individually edited the final group document. In another case (NT5), group members explicitly divided the task and assigned sections for teams to produce. (This group used the technology for their first session, and they used the same basic task approach of dividing 4A group notepad is any notepad used for recording text for the group memorandum. 12

the work in their second, NT session.). Neither of these instances suggests a pattern of individual producing of the kind we observed in the T condition. In the T condition we see a distinct tendency for groups to produce individually for extensive periods of time. All groups engaged in sporadic periods of individual work throughout the writing process. In 4 of the 5 groups, subjects drafted alone for significant periods, occasionally conferring with other group members. Three of the groups (T1, T2, & T5) made an explicit decision to work individually, either dividing the sections of the memorandum or each writing complete memoranda for later comparison and integration. For two of these groups, the decision to work individually came after they had difficulty composing as a group. For another group (T3) no systematic plan of attack emerged and members worked more haphazardly, at times on their private computers and at other times sharing the public computer. Only one group (T4) came close to producing as a group. This group tried to generate text together, but one member's ongoing efforts at revising a copy of the group's draft on his private computer fragmented the process. As we observed, extensive individual text production in the T condition reduced the amount of time groups worked together to produce text. Overall, groups spent more time actually generating text together in the NT condition. In the T condition, groups spent more time deciding what to do with individually-generated texts than actually writing together. More Revising With Technology We also sought to discover how much time groups spent revising their group documents. The logging data shows that groups engaged in significantly more revision in the T condition (29.5%) than in the NT condition (11.8%) (paired-t = 3.3, p =.03, 2-tailed.5'6). In both conditions, the vast majority of revisions were at the word or sentence level. However, in the T condition, writers also moved large pieces of text more frequently., In the NT condition wordsmithing very often preceded the actual production of text. That is, group members often discussed the wording for their text extensively prior to entering text into the 5Since only revisions of the group document were logged, this score does not reflect any revisions subjects may have made to their texts on their private computers or individual notepads. 6These scores for revisions were obtained by re-logging the videotapes of the sessions for this particular category while also viewing each group's final document and individual group members' working notes. We decided that the added information from having the documents and notes available during logging would increase the accuracy of the logging. 13

group document. This particular pattern of group producing, compared to the frequent individual producing in the T condition, may be one reason we see less frequent revisions in the NT condition. The video scenarios suggest that group members were more focused and attentive in the NT condition when revisions were being made to the group document, compared to the T condition. The fact that in the T condition everyone could see the public document and everyone had easy access to it may have made individuals feel more comfortable making changes without explicitly consulting their colleagues or getting consensus. In contrast, in the NT condition since it is more difficult for all group members to see the group document, writers may have consulted the other group members more often about potential changes before modifying text. In the questionnaires 90 percent of the subjects reported a clear preference for the computer writing tools for revising group documents. This is to be expected, considering the relative ease of revising with even the simplest word processing tool. The erasing and copying over one experiences with traditional writing tools is considerably more difficult by comparison. No Time Savings with Technology Using the computer(s) to generate their memoranda did not save the groups time overall. The average times to complete the tasks were 1 hour, 55 minutes for the T condition, compared to 1 hour, 43 minutes for the NT condition. While the amount of time spent completing the task did not vary significantly with condition, the ways groups spent their time did differ, as we have seen. Summary of Findings on the Writing Process Our comparative analysis suggests that generally groups employ different collaborative writing processes in T and NT environments. In the T condition groups engaged in less initial group planning and often began producing texts almost immediately. When groups did some initial planning as individuals, they typically did not share this planning with other group members prior to beginning their memoranda. Our video scenarios suggest that this lack of initial group planning delayed group negotiation and consensus-reaching in the T condition. Often groups did not discuss key questions about the nature of the writing situation or reach consensus about appropriate communication strategies until after individual texts had been produced. This made negotiating and consensus-reaching more difficult. In the NT condition there was almost always a great deal of negotiating prior to anyone producing much actual text. We also found that the T condition promoted more revising. Taken together, these findings indicate that technology 14

significantly alters the collaborative writing process. The differences in the writing process between conditions is illustrated in Figure 3. T Condition NT Condition Little or No Initial Group Planning Extensive Initial Group Planning Group Negotiation and Consensus-Reaching \_________________________} Sporadic Individual Producing Review of Individual Texts Group Negotiation and Consensus-Reaching Extensive Group Producing Review of Group Text D * * & Extensive Revising Limited Revising D Figure 3. Contrasting Patterns in T and NT Conditions Technology Alters Group Interaction We also find that the technology has a considerable impact on the nature of group interactions. This is consistent with findings from previous studies on computer-supported meeting environments and computer-mediated communication, which have shown that technology reduces interpersonal communication and group interaction (Kiesler, Siegel, & McGuire, 1988; Nunamaker, Applegate, & Konsynski, 1988; Poole, Holmes, & DeSanctis, 1988). In our study the amount of individual tool use, speech and group focus varied substantially across conditions. Feedback from users suggests that the technology can both enhance and inhibit communication. In the following sections we discuss these findings in more depth. 15

More Individual Tool Use With Technology In the T condition, groups usually engaged in more individual work and more private tool use, during the writing process. Comparing the logging data across conditions, the frequency of private tool use for the T condition (35.7%) was higher than for the NT condition (25.1%). Although this difference is not statistically significant, we believe it is noteworthy. As we noted earlier, in the NT condition Group 5 divided much of the drafting and revising among teams. During this teamwork, members used the individual note pads much more extensively than the other groups, which relied more on a single group notepad for developing their texts. When analyzing tool use data excluding this one exceptional group, we find that in the T condition, one or more group members used a private computer 39.3% of the time, in contrast to the NT condition, where one or more group members used a private notepad 16.0% of the time (paired t = 4.3, p <.03, two-tailed). Less Speech and Group Focus With Technology Groups engaged in speech significantly less often in the T condition than in the NT condition. In the T condition speech was logged 85.4 percent of the time; whereas, in the NT condition speech was logged 98.4 percent of the time (paired-t = -4.7, p =.009). The amount of speech in the T condition is quite high, especially given the extent of private computer use, individual work, and the placement of the display monitors on the conference table, partially interfering with group members' views of each other. Across all 5 groups, the extent to which groups achieved group focus did not differ significantly between conditions. In the NT condition groups were logged as focused 68.6 percent of the time and in the T condition 52.6 percent of the time. However, analyzing the data excluding Group 5 (the group described above as working almost exclusively in small teams and individuals in the NT condition), we find that group focus is much more frequent in the NT condition (80.3%) than in the T condition (48.8%)(paired-t = 4.05, p <.03). Contrasting Communication Effects With Technology Informal feedback from users suggests that despite the logging data results showing less speech and less group focus among group members, the collaborative technology helps group members to communicate their ideas and work together more easily. In response to questionnaire items about 16

how their groups worked together and the effects of the writing tools on one's ability to communicate his/her ideas, members from 4 of the 5 groups commented that the computers helped individuals to express ideas, to view ideas, and to have a complete exchange of individual work which would not be practical without the technology. They also reported that the computers provided a greater equality of communication among members. A counter-example to this feedback was group T4. One member dropped out of the group composing effort early and became the group's editor, privately revising a draft of the group's memo which he had transferred to his private computer. For much of the session he worked alone without telling his colleagues what he was doing. Consequently, toward the end of the session, the group realized it had generated two versions of the memoranda, one composed collaboratively by three group members and another developed individually by one group member. In the end, this situation placed the group in conflict over which document to use. This conflict was eventually resolved by integrating sections of the document developed individually into the document composed collaboratively. Comments from members of group T4 convey this conflict and the group's difficulties collaborating. The "editor" admitted in his questionnaire to "removing" himself from group brainstorming and composing. Another member complained about this, "I... didn't like that Dave didn't contribute when we were initially writing together... [he] only contributed at end... I didn't like that at all." Video evidence revealed that during the first printing of the "editor's" version of the memorandum, another member said angrily "What the hell is printing?" As the group worked to integrate the two texts, their conversation reflected a "mine/yours"' dynamic; that is, a conversation with frequent references to text as "mine", "ours", "yours," etc. Two members also complained explicitly that the T condition impaired communication. One commented, "I couldn't communicate very well. Computers were in the way physically and psychologically." Another remarked, "I never saw my ideas and [it was] harder with the computer to get them across." Summary of Findings on Group Interaction The writing process and collaborative patterns that typified the groups in the T condition suggest that the technology may both hinder and enhance group interaction. It prompts group members to begin working individually almost immediately, often without sufficient group planning, coordination and consensus. It reduces to some extent the amount of speech and focus. 17 I

However, the individual work prompted by the technology may be important in allowing group members to develop their ideas without premature pressures for consensus. As Finholt, Sproull & Kiesler (1990) noted, when there is pressure for consensus among group members, the effectiveness of individual members who may have particular expertise can be reduced. In addition, the ability to transfer work quickly and easily between the public and private computers can reduce the coordination and transaction costs (Malone, 1988) inherent in group work. The informal feedback from users supports the notion that computer capabilities provide additional channels for distributing information among group members and for merging the efforts of individuals. Technology Does Not Alter Product Quality Generally groups did not produce higher quality documents in the T condition. None of the holistic measures revealed significant differences in document quality between conditions. Group ID Tech Non tech 1 6 5 2 6 5 3 2 5 4 5 6 5 4 4 MEAN 4.6 5.0 Table 3 Holistic Scores With one exception, the documents scored in the mid-range on the seven-point holistic rating scale (1 being the lowest quality; 7 being the highest), as seen in Table 3. Mean ratings were 4.6 for the T condition and 5.0 for the NT condition. There were no high-scoring documents. Two groups (T1,T2) received slightly higher ratings on the documents they composed in the T condition, one group (T5) received equivalent ratings in both conditions, and one group (T4) received a slightly lower rating on the document they composed in the T condition. The exceptional group (T3), received a much lower rating on the document they composed in the T condition, perhaps in part because they did not finish the document in the allotted time. The other two holistic measures did not reveal any overall effects of condition on document quality. Only one of the analytic measures, the Persuasive Adaptiveness Measure, suggests an interesting difference in the documents produced in the two conditions, as shown in Table 4. This measure is a five-level scale for scoring the linguistic and rhetorical features of a document that address reader 18 - 1

concerns (see Appendix 2). A well-adapted or highly "reader-focused" document is given a score of five, whereas a document without any reader adaptation receives a score of one. Four of the five group documents composed in the T condition received an adaptiveness score one level higher than those group documents composed in the NT condition. Documents from the fifth group received the same adaptiveness score in both conditions. (Mean scores across all groups were 4.0 for the T condition and 3.2 for the NT condition). Group ID Tech Non tech 1 4 3 2 4 4 3 3 2 4 4 3 5 4 3 MEAN 3.4 3.0 Table 4 Persuasive Adaptiveness Scores While this difference in scores was not statistically significant, the fact that four of the groups scored higher on adaptiveness in the T condition suggests that the technology may facilitate reader focus. Perhaps the ability for group members to simultaneously review their document on the public screen, as well as to individually review printouts of their document, enabled them to keep their reader in mind. (We are currently analyzing the groups' dialogues during the writing process to explore in more depth the nature of their reader analysis in the two conditions). Summary of Findings on the Written Product In general, the technology did not affect the quality of the written product. Neither the holistic nor analytic measures revealed significant differences between the T and NT conditions. Group Approaches and Relationship to Document Quality As the holistic evaluation showed, two groups produced slightly better documents in the T condition and two groups produced better documents in the NT condition. We compared the work patterns of these two sets of groups, in the two conditions, in an attempt to understand how they produced the higher quality documents in each condition. Deciding how to work The two groups that produced better documents in the T condition (T1, T2) made an explicit decision to work individually and to merge their efforts later. Both groups made this decision after 19

initial periods of private and group composing interspersed, during which they expressed frustration about composing as a group. In contrast, the groups that produced lower quality documents in the T condition (T3, T4) never reached consensus on how to work and never explicitly decided to work individually in this condition. The individual work for these groups was more haphazard, with less communication among members about what they were doing as they worked individually. Group experience and technology It's interesting to note that the two groups producing inferior memos in the T condition were at opposite ends of the spectrum in terms of group experience. The most experienced group (T4) had worked together extensively over several months. (This was the group in which one member edited the "group document" throughout the session). The least experienced group (T3) had hardly worked together at all, due to sporadic class attendance. For the most experienced group, the technology disrupted their well established collaborative patterns. For the least experienced group, the technology may have made it more difficult for the members to learn how to work as a group. (In fact, one member of this group wrote that he "tuned out" and played on his private computer when he became frustrated with the group.) Tool preferences parallel performance Subjects' tool preferences are consistent with the groups' patterns of tool use. When asked which set of tools they preferred for composing, 86% of the subjects from the groups producing better memos in the T condition selected the computer tools and only 14% selected the conventional tools. In contrast, subjects from the groups producing better memos in the NT condition showed some preference for the conventional tools; 62% selected the conventional tools and only 25% selected the computer tools. (Thirteen percent reported no preference). To what extent this preference for the conventional tools would diminish as groups worked with the technology over time is an important question, especially with respect to training and experience, as we discuss below. Summary of Findings on Individual Patterns This comparison of individual patterns suggests that groups' decisions about how to work, their experience as a group and their tool preferences can influence the quality of the group product. 20

CONCLUSIONS Suggestions for Using Collaborative Computer Technology Subjects' comments during the experimental sessions indicate that they expected to find the group writing easier in the T condition and that they were surprised at some of the difficulties they encountered coordinating their use of the technology. Subjects also reported that the writing tools in the T condition diverted attention (their's and others') from the group's task. Members of several groups also commented that they had difficulty with information overload, that is, with too many ideas "thrown up on the screen" from the individual computers. Others felt that the technology provided too much freedom to edit and revise. Though the individual computers seem to beckon users to do private work, extended individual work without group negotiation and consensus-making can be premature and costly for the group. Difficulties with information overload, duplication of effort, ownership conflicts and lack of consensus are some of the problems that may arise. We've also seen how groups using the technology generate much work privately but then fail to share it among group members or delete it prematurely before everyone has had a chance to review it. From our study several suggestions emerge for how groups can avoid these difficulties and work effectively using computer support for face-to-face meetings. Following these suggestions should reduce some of the group process problems (c.f. Forman & Katsky, 1986) that often arise in collaborative projects. We also believe these suggestions are applicable to many kinds of small group tasks. Groups need to engage in initial planning as a group, regardless of whether they also pursue some kind of individual planning. When using technology designed to allow maximum flexibility in how groups work, like the Capture Lab, groups need to include this step in their collaborative process. Group members also need to make systematic decisions about when to work as a group and when to pursue individual work, and to communicate openly about the nature of their individual work. When groups do engage in significant individual work, they should develop a plan for systematically reviewing and integrating this work, to increase participation and buy-in among group members as well as to minimize members' wasted efforts. 21 --—. --- — -11.-ll~F~ --- —~-^-~~IP~~~[~~~~~~~ ~~~~~~~~ — -— U -1 -~

For groups who need or want to work face-to-face and synchronously on a problem, dividing the task and working individually some of the time can yield better quality results than working exclusively as a group, as our comparison of groups' work patterns suggests. Groups who adopt this strategy may be more satisfied with the technology as well. Directions and Questions for Future Research Our findings strongly point to the need for training that addresses the unique problems groups encounter when seeking to effectively collaborate with new technology. What kinds of benefits could we expect groups to achieve, working in a computer-supported meeting environment, given training that incorporates the kinds of suggestions we offer above, for example? How much direction do groups need in order to work effectively with this kind of technology? These remain open, important issues for further study. Experience working with the technology over time will likely be even more critical than training. As we noted initially, our subjects were at a disadvantage in the T condition, in terms of their lack of prior experience working with these new group tools. Groups do not initially understand how the technology can support their interaction. For example, groups did not know how the public screen can be a focusing tool. In their initial use of the room we saw that instead, the public screen became a diversion at times as users switched haphazardly between public and private work, without adequate communication. Additional experience will introduce users to the range of possibilities for collaborative tasks that do not exist with conventional tools. It will take time in such a new environment for a group to find its rhythm, to develop working patterns to take advantage of the tools, and to have the tool use become second nature. Longitudinal research focusing on how groups use computer-supported meeting rooms over an extended period of time is most critical in identifying the real benefits (and costs) of this technology for collaboration. An important component of this continuing research will be to provide users with a toolkit of innovative software for both synchronous and asynchronous collaboration. Discovering how and when groups choose to work together versus apart, and what tools they choose to use to accomplish their work will deepen our understanding of collaboration and hopefully provide new insights about what new technologies to develop. 22

ACKNOWLEDGEMENTS We would like to thank Mary Elwart-Keys and Andy Nolan for their technical support throughout the project, Shaul Markovitch for building our logging tool, and Joyce Massey and Israel Porat for their comments on an earlier version of the manuscript. Thanks also to Carol Mohr and the University of Michigan Computing Resource Center for their assistance in training the subjects. 23

REFERENCES Couture, B. & Rymer, J. (1989). Interactive writing on the job: Definitions and implications of "collaboration", In M. Kogen, (Ed.), Writing in the Business Professions (pp. 73-93). Urbana, IL: NCTE & ABC. DeSanctis, G. and Dickson, G. (1987). "GDSS Software: A Shell System in Support of a Program of Research," Proceedings of the 20th Annual Hawaii International Conference on System Sciences. Diehl, M. & Stroebe, W. (1987). Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of Personality and Social Psychology, 53, pp. 497-509. Elwart-Keys, M., Halonen, D., Horton, M., Kass, R. & Scott, P. (1990) User interface requirements for face to face groupware. Proceedings of Conference on Human Factors in Computing Systems, CHI '90, Seattle, WA (pp. 295-301). Association for Computing Machinery, New York. Elwart-Keys, M. & Horton, M. (1990). Collaborating in the Capture Lab: Computer support for group writing. Bulletin of the Association for Business Communication, expected June, 1990. Forman, J. (forthcoming). Computing and collaborative writing: Issues for the 90's. To appear in G. Hawisher & C. Selfe, Questions for the 1990s: Scholarship and research on computers in the next decade. Urbana, IL: NCTE. Forman, J. & Katsky, P. (1986). The group report: A problem in small group or writing processes? The journal of business communication, 23:4, 24-35. Finholt, T., Sproull, L., & Kiesler, S. (1990). Communication and performance in ad hoc task groups. In J. Galegher, R. Kraut, & C. Egido, (Eds.), Intellectual teamwork: Social and technological foundations of cooperative work. Expected, May 1990. Hillsdale, N.J.: Lawrence Erlbaum Associates. Haas, C. (1989). Does the medium make a difference? Two studies of writing with pen and paper and with computers. Human-Computer Interaction, 4, pp. 149-169. Hayes, J. & Flower, L (1980). Identifying the organization of the writing processes. In L. Gregg & E. Steinberg (Eds.) Cognitive processes in writing, pp. 3-29. Hillsdale, N.J.: Lawrence Erlbaum Associates. Jewell, L. & Reitz. H. (1981). Group effectiveness in organizations. New York: Scott, Foresman and Company. Kiesler, S., Siegel, J., & McGuire, T. (1988). Social psychological aspects of computermediated communication. In I. Grief (Ed.), Computer-supported cooperative work: A book of readings (pp. 657-682). San Mateo, CA: Morgan Kaufman Publishers, Inc. Malone, T. (1988, October). What is coordination theory? Paper presented at the Coordination Theory Workshop of the National Science Foundation, Massachusetts Institute of Technology, Cambridge, MA. 24

Mantei, M. (1988). Capturing the capture lab concepts: A case study in the design of computer supported meeting environments. In I. Grief (Ed.), Proceedings of the 2nd Conference on Computer-Supported Cooperative Work (pp. 257 —270). Association for Computing Machinery, New York. Nunamaker, J.F., Applegate, L., Konsynski, B. (1988). Computer-aided deliberation: Model management and group decision support. Operations Research, 6, pp. 826 -848. Poole, M.S., Holmes, M., & DeSanctis, G. (1988). Conflict management and group decision support systems. In I. Grief (Ed.), Proceedings of the 2nd Conference on Computer-Supported Cooperative Work (pp. 227 —242). Association for Computing Machinery, New York. Rein, G. & Ellis, C. (1988). The Nick summer experiment: A field study on the usage of meeting support technology by software design teams. MCC Technical Report, Number STP-018-88, Software Technology Program. Rogers, P. (1989, November). "Textual Analysis to Identify Assessment Criteria," Presentation at the International Convention of the Association of Business Communication. Smith, J. & Lansman, M. (1989). A cognitive basis for a computer writing environment. In B. Britton & S. Glynn (Eds.), Computer Writing Environments: Theory, Research, and Design. Hillsdale, NJ: Lawrence Erlbaum Associates. Stefik, M., Foster, G., Bobrow, D., Kahn, K., Lanning, S., and Suchman, L. (1987). Beyond the chalkboard: Computer support for collaboration and problem solving in meetings," CACM 30 (1): 32-47. Vroom, V., Grant, L., & Cotton, T. (1969). The consequences of social interaction in group problem solving.", Organizational Behavior and Human Performance, 4, 77-95. 25 ---------— ~ —~ --- —----- — —-"1-~I --- —- I

Appendix 1 Writing Tasks CaseA Background As Directors of Personnel, you make all the major managerial hiring decisions for Nelson's Hardware, a chain of hardware stores in the Midwest. Founded in 1966, Nelson's is a strong organization, which consists of 82 stores in Wisconsin, Illinois, and Michigan. In 1984, Nelson's merged with a large home supplies distributor, James & Turney (J&T). At that time, J&T supplied funds and mandated expansion. As a result, Nelson's grew quickly. This year alone, Nelson's opened 22 new stores. Until the merger with J &T, Nelson's workers were like family. The well-known company policy was to consider present employees first for any promotions. Store managers were regularly recruited from within the company. With rapid expansion, however, you recently recruited 10 managers for new stores from outside the company. After much consideration, you also replaced two old-line managers, Jane Kitz and Bob Craig, with outsiders. As longtime Personnel Directors you all support Nelson's policy of promoting from within; however, none of you is overly concerned about outside recruiting when no old-line employees qualify for advanced positions. Rumors Among Old-Line Employees In recent months, all of you have become aware that the old-line employees deeply resent your outside recruiting. You've recently reviewed a memo from four of Nelson's most experienced store managers. They said the "new hiring policy" was hurting the company. They outlined three objections to hiring outsiders. 1. New store managers are being hired from outside the company without due consideration of old-line employees. 2. Old-line store managers have no job security and are being systematically replaced by outsiders. (For instance, Jane Kitz and Bob Craig were recently replaced.) 3. Store managers hired from outside are not adequately qualified because they do not understand the company's history and they do not know Nelson's employees. In your recent directors' meetings you've discussed the fact that you have a serious communication problem. Old-line employees have misconceptions about the company's growth and your hiring practices. You've all heard store managers making statements like, "As long as Nelson's is going to bounce me in the near future, I might as well leave while I can make a good connection." Employee morale is low and sales are down. Several store managers are threatening to move to competitors. 26

Appendix 1, cont. Case B Background As the Sales Management Team for Sportech, Inc., you oversee the selling of all Sportech products and services. Founded in 1961, Sportech is a small but competitive company that produces and sells sport training equipment. Over the years, Sportech has developed a reputation for quality products backed by an excellent sales and service staff. Growth and competition in the health and fitness industry have contributed to Sportech's profits. Additionally, much of the company's success is attributed to the knowledge and skill of Sportech's sales staff. As you know from personal experience, potential members of Sportech's sales staff are required to successfully complete Sportech's rigorous, four-week Sales Training Program (STP). Potential sales personnel join the company with excellent athletic backgrounds, but almost no formal sales and communication training. STP is designed to develop interpersonal and selling skills. Sales trainees learn everything from opening a conversation with a prospective client to closing a sale. As the Sales Management Team, you attribute much of your sales division's effectiveness to the STP. Moreover, the success of new sales personnel brings increased profits to the the entire sales staff under Sportech's profit-sharing plan. Proposal to Eliminate the Sales Training Program (STP) At a recent meeting, Sportech's Executive Committee, headed by the company's new president Sue Walker, announced their decision to "cut the fat out of Sportech." Speaking for the Committee, Walker stated, "We must focus our resources on state-of-the-art technology and eliminate non-essential, costly programs such as the STP. Walker outlined three reasons why the STP should be eliminated: 1. Sales techniques are best learned through immediate and direct experience with Sportech products and customers. 2. Sportech's Sales Management Team can provide effective on-the-job training for new sales personnel. The Sales Management Team can set aside time to train new personnel in reporting procedures and sales techniques. 3. STP is not cost or time effective, and Sportech's resources are needed for new technological developments if the company is to remain competitive. After the meeting you expressed your concerns, as members of the Sales Management Team, about the decision to drop the STP. Walker seemed to interpret your concerns as a challenge to her leadership. "We developed this resource redistribution plan after months of thorough analysis," she said. "All members of Sportech's Executive Committee support it; however, if you disagree you may discuss your objections in a memorandum." You support Walker's innovative leadership. At the same time you believe STP is essential for Sportech's continued success. 27

Appendix 2 Persuasive Adaptiveness Measure Adaptation of Delia, Kline & Burleson Measure of Persuasive Adaptiveness The Persuasive Adaptiveness Measure outlined below is a five-level hierarchy designed to score the extent to which a writer identifies his/her managerial decisions (specifically his/her conclusion or recommendations) with the needs and concerns of his/her reader(s). At the low end of the hierarchy are those documents in which the writer does not even state his/her conclusions or recommendations and thus completely alienates the reader. Such documents are awarded a score of zero. At the high end of the hierarchy are those documents in which the writer presents his/her conclusions or recommendations by focusing the document around the advantages the reader gains from accepting them. Such "reader-focused" documents are awarded a score of five. The Persuasive Adaptiveness Measure is not designed to score "tone" or emotional appeals per se, but rather seeks to reward a writer's very deliberate, analytical attempts to achieve his/her managerial goals by genuinely identifying them with reader needs and concerns. 0 Writer conclusion(s)/recommendation(s) not clear 1 Unelaborated statement of writer conclusion(s)/recommendation(s) 2 Statement of writer conclusion(s)/recommendation(s) with need or usefulness suggested 3 Some elaboration on need or usefulness of writer conclusion(s)/ recommendations(s) 4 Elaboration on need or usefulness of writer conclusion(s)/ recommendations(s) plus minimal dealing with possible reader objections or concerns about accepting those conclusion(s)/ recommendations(s) 5 Elaboration on reader advantages of accepting writer conclusion(s)/ recommendation(s) 4 28

r J. F '.