Research Support School of Business Administration July 1994 WHAT DOES THE GMAT ANALYTICAL WRITING ASSESSMENT BRING TO MANAGEMENT EDUCATION AND COMMUNICATION? Working Paper #9407-19 Priscilla S. Rogers University of Michigan and Jone Rymer Wayne State University

I

WHAT DOES THE GMAT ANALYTICAL WRITING ASSESSMENT BRING TO MANAGEMENT EDUCATION AND COMMUNICATION? Priscilla S. Rogers University of Michigan School of Business Administration 701 Tappan Street Ann Arbor, Ml 48109 313-764-9779 Jone Rymer Wayne State University School of Business Administration Home: 3701 Middleton Ann Arbor, Ml 48105 313-996-1521 FAX 313-662-5132 Email: jrymer@cms.cc.wayne.edu This paper is currently under journal review. Authors' Note: We thank Jane Thomas and David L. Williams for providing us with their insights on an earlier draft. We contributed equally to this article and have listed our names in alphabetical order.

WHAT DOES THE GMAT ANALYTICAL WRITING ASSESSMENT BRING TO MANAGEMENT EDUCATION AND COMMUNICATION? [ABSTRACT] This article provides a critical analysis of the Analytic Writing Assessment (AWA) added to the Graduate Management Admission Test (GMAT) in 1994 -95. Historically, the AWA is significant for the recognition it brings to writing as a high-level, essential management ability. Functionally, the AWA will provide a new indication of MBA applicants' academic abilities for graduate study, useful for admissions. However, as a test of writing abilities, the AWA has limitations. In assessing analytic writing, a type of general classroom writing, the AWA ignores disciplinary and managerial writing tasks important for MBA study and also ignores management writing for the workplace. As a diagnostic tool, the holistic scores and unmarked essays do not readily help schools to identify applicants who need further work in writing. Thus, the AWA makes a contribution to management education, but it does not fulfill some explicit expectations of GMAT-subscribing schools, especially in terms of predicting competency in managerial writing.

1 WHAT DOES THE GMAT ANALYTICAL WRITING ASSESSMENT BRING TO MANAGEMENT EDUCATION AND COMMUNICATION? In the fall of 1994, the Graduate Management Admission Test (GMAT) introduced a new section, called the Analytical Writing Assessment (AWA). Characterized by the president of the Graduate Management Admission Council (GMAC), William Broesamle (1993), as "the most significant change in the GMAT since it was first administered," the AWA adds writing performance to a standardized multiple-choice test for the first time. Test-takers write two essays which are evaluated to assess critical thinking and writing ability, in addition to answering the quantitative and verbal questions of the GMAT. The introduction of the AWA into the GMAT places those involved with management education and communication at a critical juncture. Eventually, GMAT-subscribing schools across the country will receive writing scores for thousands of MBA applicants; these scores will influence admissions, placement, and program decisions. Management educators will have to determine whether they should use the results of this new writing test, and if so, how; they must also consider its ramifications for management education and communication.1 The GMAC Analytical Writing Assessment raises a surprising number of important diagnostic and curricular questions, including the following: - How useful is the test for MBA programs? - Will the AWA identify applicants whose writing is inadequate for graduate work in management? - Should individuals who score low be provided with special courses or tutoring? - Should existing programs be altered based on the results of the test? - Will the AWA identify individuals who will perform well as writers in their management careers? As business school administrators and faculty address these and other

GMAT Writing Assessment 2 questions, the influence of the new GMAT writing test will be felt on MBA education. On the one hand, the AWA may elevate writing as a vital management competency, increasing its role in the overall MBA curriculum and creating demand for new workshops and tutorials. On the other hand, the writing test may prompt modifications to communication courses and writing requirements, perhaps replacing local assessments and waiving core courses for selected students. More significant, the new test may come to be regarded as a standard for management writing, thus influencing both the perceptions and practices of communication in business schools. The GMAT analytical writing test does, indeed, place us at a critical juncture, a juncture that is yet little understood. The purpose of this article is to analyze the new GMAT writing assessment and its potential impact on management education. If management communication scholar-teachers become knowledgable about the AWA — especially regarding its usefulness and limitations —they will be able to facilitate the test's appropriate application and actively participate in the important conversation that will help shape the future of managerial writing. We begin by examining the AWA's historical significance, placing the test in the context of developments in management education and in testing. Next, we analyze the test's functional significance and compare the results it will yield with potential users' expectations. Here we grapple with critical questions regarding the nature, relevance, and diagnostic usefulness of the test. Our analysis shows that the AWA contributes to management education by acknowledging management writing as a significant competency and as a high-level, complex ability. The AWA should function as a useful new indicator of MBA applicants' academic abilities for graduate study in management, scores useful for MBA admissions. However, as a writing assessment, the

GMAT Writing Assessment 3 AWA has some serious shortcomings. It tests general academic writing that only partially reflects writing required for management studies and that is largely irrelevant to management writing in the workplace. Neither does the AWA provide specific diagnostic information that can assist students in strengthening their writing abilities. Overall, the AWA is an academic writing assessment that will meet only some of the expectations of management educators. WHAT IS THE HISTORICAL SIGNIFICANCE OF THE GMAT ANALYTICAL WRITING ASSESSMENT? Traditionally, business schools have not been strong advocates for testing writing. Many administrators have privileged the GMAT quantitative score over the verbal score (O'Reilly, 1994) —which tests reading comprehension, critical reasoning, and ability to identify syntactic and usage errors. In fact, a decade ago a joint report by GMAC and the Educational Testing Service (ETS) recommended adding a writing test to the GMAT (Hecht & Alloway, 1984), but it was not adopted, reputedly because the benefits of a writing test did not seem to warrant the high costs and many difficulties of implementation, including concerns of business faculty about subjectivity in evaluating writing.2 Over the past few years, this reticence toward testing writing changed, so that when GMAC proposed adding an essay to the GMAT, the response of business schools was very positive (Bruce, 1993). In October of 1994, the GMAT Analytical Writing Assessment was first implemented, requiring more test-taker time and money, as well as reducing time devoted to the quantitative and verbal sections of the test. Now, in addition to completing the objective GMAT sections, individuals must write two short essays in 1 hour.3 The essays are assessed by multiple evaluators to produce a single overall "AWA

GMAT Writing Assessment 4 Score" for each test-taker. Subsequently, when an individual applies to MBA programs, his or her AWA score and essays are provided to each school. So what motivated the introduction of the GMAT Analytical Writing Assessment now? We believe that two important trends —both responding to demands for accountability to the various constituencies of business schools — created a climate of opportunity for the GMAC to implement the new test at this time: 1) the on-going effort to reformulate MBA education, an effort that acknowledges communication as a critical managerial ability; and 2) the movement toward performance assessment in testing, a method of testing that elevates writing from a lower-level skill to a higher-level ability. Exploring these trends reveals the importance of the GMAT analytical writing test for management education. WRITING AS A CRITICAL MANAGEMENT ABILITY The emergence of the GMAT writing assessment is associated with the reformulation of management education in response to its larger environment. Prompted by the business communities' increased concern for selecting the best managers to hire and retain, for serving customers, and for competing in global markets, this reinvention of the MBA directly responds to complaints about graduates' communication abilities and "people" skills.4 Numerous stakeholders in management education have issued appeals to business schools, as reported by the GMAC Admissions Commission which challenged MBA administrators to ensure "that graduates are able to deal effectively with other people, both in person and in writing," and which suggested the development of "new assessment devices that begin to provide support for broader admissions criteria" (1990, pp. 19, 41). GMAC explicitly links the test with "new management demands" which include writing (GMAC, nd, p. 1), and

GMAT Writing Assessment 5 points to the overwhelming support (90 percent) the AWA received from business school deans and MBA program administrators that GMAC surveyed in 1993 (Bruce, 1993, p. 24).5 Certainly, there is a felt need, both within the academy and the business community, to do something about management students' writing abilities. By introducing writing assessment into the prestigious national GMAT and by securing the blessing of business school administrators, GMAC brings wide attention to writing as a management ability. This is significant because rather than treating writing as a central part of graduate study in management, business schools have typically marginalized writing, frequently "covering" it in the core program by assigning writing as a means of demonstrating knowledge of course content, and relegating actual instruction in writing to skill-based offerings, remedial workshops, and electives. The GMAT writing assessment is particularly timely given the focus on oral communication in research on managers (Smeltzer & Thomas, 1 994) and the fact that the most visible changes in management education to date have focused on oral communication. General management curricula include a range of communication subjects: interpersonal and group communication, workgroups, meeting and conflict management, decision-making, leadership, corporate vision, and motivation —all of which tend to be taught as oral communication. For example, teamwork is perceived as an oral activity (Schwindt, 1990). Teams may be encouraged to use writing as a tool for recording decisions (Scholtes, 1992) and brainstorming (Moore, 1 994), but seldom is written communication recognized as central to the group process (Rogers & Horton, 1992). Moreover, although some MBA programs have added communication courses which include writing, there is no curricular trend to establish writing

GMAT Writing Assessment 6 instruction for management. In fact, whereas some graduate schools of business have added core courses featuring written and oral communication (Sloan at MIT), some have merged required communication courses with other types of courses (Duke), and some have completely eliminated such courses (Harvard). Writing is also taught in workshops, communication electives, managerial writing assessment programs, and by integrating communications components into functional courses and special projects, but these offerings vary tremendously school-to-school. There simply is no benchmark for the way writing should be taught in MBA programs (Munter, 1989a, 1989b, 1990; Shelby, 1994). The variety of communication offerings confirm that business schools are aiming to comply with the accrediting directive to "include written and oral communication as an important characteristic" of MBA studies (American Assembly of Collegiate Schools of Business, 1993, pp.17-19), but their efforts also reveal that, as a community, business school faculties have not determined how writing fits into graduate study of management. Although there is much lip service to the importance of writing and the notion that it should be included in MBA studies, somehow and somewhere, schools lack consensus on the place of writing instruction in management education. The introduction of the GMAT Analytical Writing Assessment could change this. The AWA provokes educators to view writing as an essential ability for graduate studies in management, an ability that should be systematically integrated into MBA programs. WRITING AS A HIGH-LEVEL ABILITY The AWA is a performance-based assessment that redefines writing as a high-level ability rather than a low-level skill. Part of a broad educational trend toward testing competency through performance, the AWA requires applicants

GMAT Writing Assessment 7 to demonstrate ability to write by writing instead of by merely answering questions about writing. To complete the AWA, test-takers compose whole pieces of discourse from scratch, a task that requires developing ideas, formulating plans, and organizing content, as well as drafting, revising, and editing paragraphs, sentences, words, grammar, and mechanics. In short, the AWA tests a wide range of cognitive abilities, procedural knowledge of composing, as well as the ability to apply many language and textual skills. As a performance-based test, the AWA is a "direct" test of writing, in contrast with objective or "indirect" tests, such as the GMAT Verbal. Although indirect tests have been the traditional way to assess writing ability, such tests do not require writing at all; instead they consist of multiple-choice questions which measure passive knowledge of textual skills such as diction and syntax, conventions of Standard Written English, and reading skills. Although indirect tests are easy and inexpensive to administer and score reliably, they send misleading signals about the nature of writing ability, equating it with editing rather than with the complexities involved in actually composing text. Because indirect tests lack face validity for test-takers, and because such tests tend to correlate only modestly with any actual writing performance or pertinent academic performance, indirect tests are largely being replaced by direct tests of writing (White, 1993, 1994; Williamson, 1993).6 Despite the difficulties of testing writing directly, most national tests, including professional school tests, now use performance-based or direct assessments of writing. Performance-based writing tests, such as the Medical College Admission Test (MCAT), established not only a favorable environment for the GMAT, but also model methods and practices which strongly influenced the design of the AWA.7 The key aspect of these performance-based testing procedures adopted by GMAC is holistic scoring, which is based on the notion

GMAT Writing Assessment 8 that the abilities reflected in a complete piece of writing, or whole discourse such as an essay, are more than the sum of all a writer's sub-skills. By refining holistic scoring procedures over many years, researchers eventually achieved acceptable levels of agreement among evaluators, making it possible to reliably rate writing as whole discourse, even in large-scale assessments. Multiple evaluators, selected, trained, and managed carefully, can score each test paper with a single score representing its overall quality, with reasonable consistency and efficiency, thereby containing costs (White, 1994). In fact, it is the development and refinement of holistic scoring that has transformed writing assessment into a reliable and economical enterprise, making the AWA a feasible option for business school constituencies and thereby extending largescale performance assessment of writing to a new population, potential management students.8 Local performance assessment of management students' writing has been featured for many years in MBA assessment programs in some business schools, notably at the Stern School of Business at New York University and the University of Michigan Business School, and most recently at Harvard Business School.9 Although these local assessments have been successfully used as diagnostic instruments and integrated into MBA curricula, they are not widely known and do not form a standard component in the delivery of communications training in MBA programs. By introducing the performance assessment of writing into a national test, the GMAT brings performance assessment of writing to the attention of all the stakeholders in management education. Being a very public instrument, which will touch the lives of many individuals in various ways, the GMAT Analytical Writing Assessment may transform the way writing is regarded in management education by establishing

GMAT Writing Assessment 9 writing as a higher-level ability involving thinking and creating meaning, an ability warranting attention and resources. FUNCTIONAL SIGNIFICANCE: WHAT DOES THE GMAT BRING TO PERFORMANCE ASSESSMENT FOR MANAGERIAL DEVELOPMENT? The GMAT writing assessment has great potential to influence management education. According to GMAC, [The Analytical Writing Assessment] is an additional tool to help business schools select the best applicants for admission to graduate management programs, and it is a diagnostic aid for determining whether prospective and accepted students need specific work to develop communication skills" (GMAC, nd, p. 1) Business schools strongly supported both of these objectives, 91 percent for admissions and 88 percent for diagnostic purposes (Bruce, 1 993, p. 10), but whether the test will fulfill schools' specific expectations is questionable. In this section we examine GMAC's stated purposes, comparing them with business schools' expressed needs for a GMAT writing test, as surveyed by GMAC (Bruce, 1993). Beginning with the admissions function, we consider what the AWA test does and does not measure and its usefulness for selecting the fittest among MBA applicants. Next, we discuss how the AWA test is scored and analyze its usefulness as a diagnostic tool. Finally, we examine an explicit expectation of business schools that goes beyond GMAC's announced scope for the writing test, namely, serving the needs of the business community by facilitating MBAs' writing competence as managers. ASSESSING ACADEMIC WRITING ABILITIES First and foremost, GMAC intends the AWA as "an additional tool to help business schools select the best applicants for admission to graduate management programs" (GMAC, nd, p.1) and this admissions function garnered the support of many for the new writing test (Bruce, 1993, pp. 19, 28).

GMAT Writing Assessment 10 Results of GMAC's 1993 survey show that a majority of business school respondents (55 percent) expected the AWA would be "very useful" in evaluating applicants' abilities for management studies. Although schools across the board responded that the writing test would be at least "somewhat useful," respondents from elite business schools were strongly in favor of the AWA scores to facilitate admissions selections, with "top" MBA programs supporting the test for that purpose exclusively.10 These survey results suggest that elite programs want the writing test to help differentiate among applicants who have high overall GMAT scores, as well as outstanding records. In summary, potential users believe that the AWA will bring something new to the admissions process. Based on theory and research on writing and writing assessment, our analysis suggests that the AWA should provide GMAT subscribers with a new indicator of applicants' abilities for academic work. Research on other professional school tests, for example, shows that an essay measures something different than objective sections, so the AWA should fulfill GMAC's promise to enhance the value of the GMAT (Koenig & Mitchell, 1988, p. 26). However, the AWA may not be as fully relevant to writing in the MBA curriculum as GMAC claims or business school administrators expect because the essay test measures ability in only one general type of "school" or classroom writing. What is the Analytical Writing Assessment? The AWA requires testtakers to write two essays in response to two prompts, one labeled "Analysis of an Issue," and the other "Analysis of an Argument," as shown in the sample prompts in Figure 1. Though labelled differently, the differences between the two tasks are slight, with the first offering writers an issue to develop their own ideas, and the second focusing writers on critiquing an argument

GMAT Writing Assessment 11 presented in the prompt. Both the "Issue" and "Argument" prompts elicit the same genre, the analytical essay, and scores for the two essays are averaged into a single analytical writing score. 1 /insert Figure 1 about here./ True to its name, the Analytical Writing Assessment is clearly designed to test analytical writing. The AWA Scoring Guide (GMAC, nd, pp. 10-15) describes the overall characteristic for evaluating an essay as the effectiveness of the analysis. Each score on the AWA 6-point scale is described in terms of specific criteria reflecting this focus on analysis; evaluators are asked to assess how well the essay analyzes ideas, presents a point of view, develops reasons and examples, and organizes the material logically 1 2 In composing an analytical essay, a writer is required to evaluate ideas and develop a critical perspective on them, supporting this perspective with a logical presentation of points (Durst, 1987). Thus, the AWA asks the testtaker to create meaning by assuming a point of view and providing reasons for it —in effect, to tell something "new." This contrasts sharply with expository writing, which simply involves explanation or a retelling of what is already known.13 On the other hand, the AWA prompt does not call for formal argumentation with a thesis and supporting evidence (Toulmin, 1958). Although the AWA is referred to in GMAC publications as "present[ing] logical arguments" (GMAC, nd, p. 1), analytical writing simply requires the test-taker to analyze an issue or argument presented in the test, and to support that analysis with a logical presentation of details, such as examples drawn from personal experience, observations, or reading (GMAC, nd, p. 3). What is measured by the Analytical Writing Assessment? According to GMAC, "the AWA assesses the ability to think critically and to communicate

GMAT Writing Assessment 12 complex ideas," an ability that is "important to academic performance" (GMAC, nd, p. 2). For some time there has been an assumption that writing functions as a means for thinking and learning complex content in any subject area (witness the writing-across-the-curriculum movement), and recently some research has confirmed empirically that writing —especially certain types of writing —can elicit critical thinking. Using protocol analysis and other investigative methods, researchers have demonstrated that analytical writing involves a wide range of reasoning operations —interpreting, hypothesizing, evaluating, integrating and reconceptualizing ideas, including meta-cognitive procedural abilities critical to problem solving (Ackerman 1 991; 1993; Durst, 1987; Penrose, 1992). All types of writing do not require the same critical thinking abilities, however. In contrast with such typical classroom modes as exposition, description, and narrative, analytical writing is much more cognitively demanding. For example, research shows that when assigned analytical tasks, less proficient writers tend to respond with simpler writing modes, such as narrative or exposition (Durst, 1987). Given this fact, an analytical writing test can be an effective means for differentiating individuals with highly developed cognitive abilities who can restructure information and produce new knowledge, from those who can simply reproduce information by preparing an expository essay or a narrative (see Schumacher & Nash, 1991). However, in order for analytical writing to be used to measure cognitive abilities, test-takers.must have adequate time to engage in "extended" review, reflection, and revision as integral components of the composing. Research shows that some capable writers may not reveal their full abilities in a performance assessment where the strictly limited time and artificial conditions (including the prompt itself) severely constrain the naturalistic composing

GMAT Writing Assessment 13 processes (Koening & Mitchell, 1988; Ruth & Murphy, 1988; Webster & Ammon, 1 992). For those test-takers who perceive the GMAT writing test as an opportunity for high-level analysis and who can quickly respond to the prompt, engage in critical thinking on the particular issue, and execute a draft — for those test-takers the AWA should provide an opportunity to demonstrate academic abilities for graduate work. However, for at least some of those testtakers who cannot do all of these things in the 30 minutes allowed for each essay, the AWA may erroneously suggest that they lack academic ability. What is not measured by the Analytical Writing Assessment? Designing the AWA to test analytical writing, (which GMAT's constituents endorsed as a credible mode and which research presents as an appropriate vehicle for measuring academic abilities), GMAC followed the principle that a performance assessment must select the most relevant writing genre for the circumstances (Penrose, 1992;). Despite GMAC's claims for their choice of analytical writing, however, business school constituencies should not assume that the AWA fully represents writing for management education. Although the kinds of writing tasks assigned in MBA curricula have not been systematically defined by researchers, preliminary studies suggest that MBA writing, like all advanced classroom writing, is far more complex than the cognitively-focused analytical writing task used for the GMAT writing test. Common knowledge of MBA assignments (especially case analyses) tends to support the importance of writing which sounds similar to the analytical writing required by the AWA, but MBA coursework also involves other kinds of writing related to the practice of management, especially argumentation in which a student must defend a recommendation with evidence designed for specific audiences and circumstances. Moreover, recent ethnographic research suggests that the writing tasks assigned to students in MBA programs are more complex than

GMAT Writing Assessment 14 previously assumed or than the rubric "analytical writing" covers (Forman & Rymer, 1993; Freedman, Adam, & Smart, 1994). The AWA tests school writing that is not specific to graduate management education. The topics assigned in both the Issue and the Argument tasks cover matters of "general interest," which might include some business-related questions, but do not, according to GMAC, presuppose any business knowledge. Moreover, they require the application of personal experience rather than social knowledge in which business problems are typically immersed (GMAC, nd, p. 3). The AWA raters are instructors who teach and evaluate writing in a variety of academic departments, not just business schools,14 and the test, as we have seen, is a standard analytical essay, evaluated according to general criteria of classroom writing. The analytical essay defined by GMAC is, in fact, the most common task for general writing assessments in both schools and colleges. Often called the "essayist register," this school writing is not discipline-specific. Its central characteristics are its logic and clarity; it is self-contained, isolated from the context of people interacting. The writer and the audience are largely irrelevant, replaced by idealized versions of both as "rational minds," rather than as identified human beings (Farr, 1993). While general analytical writing has some pertinence to management education in indicating academic abilities, it does not target management education or management writing. Moreover, in contrast to the GMAT analytical essay, management classroom writing is influenced by the specialized disciplinary writing in various functional business fields, such as finance, human resources, and real estate (see Driskill, Ferrill, & Steffey, 1992). The writing within an academic discipline —especially the way it develops and disseminates new knowledge in a field —is not only cognitive (as is analytical writing), it is also social (Jolliffe,

GMAT Writing Assessment 15 1988). In fact, the more advanced a writer becomes in a discipline, the more domain-specific becomes his or her writing abilities, and the more significant becomes the social dimension (Carter, 1990). Researchers studying writing in many different fields, including writing directly associated with business disciplines (Devitt, 1991; McCloskey, 985; Mcsaac & Aschauer, 199 M0; and see Olsen, 1 993), have established that writing within each discipline is bounded by local knowledge and practices which are influenced by the particular purposes, values, and contexts of the academic community involved. An apprentice member of a particular disciplinary community begins learning to think, to speak, and to write as an active participant in that community, as he or she interacts with expert members (Carter, 1990; Russell, 1991). In MBA courses, for example, students are sometimes asked to play managerial roles requiring them to write from the perspective of a particular discipline, acting as a marketer, financial analyst, human resources manager, operations specialist, etc. These exercises require students to apply the disciplinary language of a particular course to a specific management situation outlined in a case or presented in a project or field-study report. Still, disciplinary writing can remain "invisible" to academic practitioners, with its characteristics not explicitly defined, nor directly taught (Russell, 1991). Faculty in many disciplines frequently assume that their students' writing is merely a vehicle for reflecting thinking about course content and showing that the material has been learned; writing well on assignments amounts to being logical, clear, and correct. It would not be surprising, therefore, if management school faculty and administrators viewed a kind of writing called "analytical," which requires the analysis of an issue or argument, as relevant to the MBA writing tasks assigned in MBA courses. Though only a few respondents to the GMAC survey mentioned the significance of "analytical

GMAT Writing Assessment 16 writing" in MBA courses as a reason for supporting the AWA, when they were offered a brief description of analytical writing, 76 percent readily confirmed that it was "very important" in their academic programs (Bruce, 1993, pp. 8, 19-20). Faculty should not assume, however, that the AWA indicates how well students will perform on written assignments in MBA courses. Just because students can produce effective analytical writing well does not mean that they can do disciplinary writing. General cognitive knowledge and problem-solving abilities can only go so far; domain-specific knowledge is necessary to be an expert writer (Carter, 1990). Although faculty may conclude that students who cannot write well in advanced classes need remedial work in writing, often these students simply do not know the disciplinary genre in question (Jolliffe, 1988; Russell, 1991). For example, Forman's (1989) research on MBA fieldstudy reports demonstrates that MBA students who were proficient in other types of writing, such as technical manual writing, were unable to do an effective job of writing strategic reports for management. Without explicit instruction, student knowledge and ability in one genre only transferred in part to the new mode, and in some respects it represented a barrier to succeeding in the new because they did not recognize that the methods and standards for "good writing" were not the same in both. Moreover, despite common beliefs that writing comprises basic skills that readily transfer across situations, there is no global concept of "good writing." Just as the rules, expectations, and standards all differ to some degree by the type of discourse, psychometric studies show that different tasks or genre call forth different cognitive abilities and skills from the writer (Quellmalz, Capell, & Chou, 1982). Studies of writers' development show that only those who possess a specific cognitive ability (say, classifying) can perform well on a

GMAT Writing Assessment 17 writing test demanding that ability, and the extent to which the ability is required depends on the mode of writing (Webster & Ammon, 1994). Many large-scale writing assessment studies clearly demonstrate this key point: writers' abilities differ according to the task or genre, and different tasks demand different abilities (Applebee, Langer, Mullis, Latham, & Gentile, 1994; Engelhard, Gordon, & Gabrielson, 1991; Purves, 1992.). In short, by asking test-takers to produce a piece of analytical discourse, the AWA cannot predict performance in all modes of writing, especially more advanced disciplinary genres. What can we conclude regarding the value of the AWA for testing the writing abilities of applicants for graduate management programs? GMAC can claim user-support for its selection of analytical writing for the GMAT, and business schools can expect the AWA to predict critical thinking abilities. The AWA will also predict performance in analytical writing, but it will not necessarily measure abilities in other types of writing required in MBA programs. Obviously, in a national writing assessment of a very diverse group of applicants, some of whom lack business experience or awareness of North American business culture, it would be difficult —though not impossible —to design a prompt more directly relevant to management education (cite Ruth & Murphy and do a note on prompt research). What is important is that in interpreting the AWA scores and using test results, management administrators and faculty do not assume that the AWA fully tests the writing abilities necessary for MBA studies, or that high AWA scores mean that an applicant has all the relevant abilities for writing as a graduate student in management. DIAGNOSING NEEDS FOR FURTHER WORK IN WRITING According to a GMAC brochure on the AWA, the writing test will also be

GMAT Writing Assessment 18 useful as "a diagnostic aid for determining whether prospective and accepted students need specific work to develop communication skills" (GMAC, nd, p. 1). GMAC suggests that the AWA will facilitate diagnostic decisions, allowing schools to direct accepted applicants to tutorial services, communication workshops, and courses, and to identify those for whom communication requirements could be waived. GMAC's survey results suggest that GMAT users registered their strongest support for this diagnostic objective, with 62 percent endorsing the test as "very useful" for "determining the student's need for additional work in written communications"(Bruce, 1993, p. 10). In replying to an open-ended question, the respondents most frequently said they supported the AWA "as a diagnostic instrument that would be useful in advising and course placement" (Bruce, 1993, p. 23). Actually using the AWA to diagnose students' communication capabilities and needs will not be so easy, however. It will require a commitment of money, time, and expertise from individual schools —a commitment that administrators and other survey respondents may not have fully anticipated when they endorsed the new test. GMAC provides participating schools with applicants' AWA scores and essays but makes it abundantly clear that interpretation of these data is strictly a local matter (GMAC, p. 9). This fact raises a critical question: What procedures must schools put into place if they want to employ the AWA for diagnostic purposes? To address this question, we describe the nature of the AWA scores and essays that GMAC will provide to schools and then examine some of the complexities involved in interpreting those scores and essays for diagnostic purposes. What do the A WA scores mean? For each MBA applicant, schools receive a single AWA score ranging from a low of 1 to a high of 6 (e.g. "AWA

GMAT Writing Assessment 19 4.5"). These scores are averages of the holistic scores given to the two essays. 5 An evaluator awards a holistic score reflecting his or her overall impression of the essay's quality based on the specific criteria established for the AWA. However, because there are multiple criteria, "responses receiving the same score may differ in approach or in the particular mix of writing features" (GMAC, nd, p. 6); that is, papers receiving a score of "4" will not necessarily have similar strengths and weaknesses. Throughout the rating process, evaluators only record their holistic scores. While they apply the specified criteria to form their impressions of individual essays, they do not record their judgments, nor do they mark the essays in any way. Consequently, when schools receive applicants' scores and the unmarked copies of the essays, it is impossible to determine which specific characteristics led to the AWA score or what deficiencies were displayed in an individual's writing without reading and analyzing the essays again. How useful are the A WA scores for diagnostic purposes? Employing the AWA scores and essays for diagnostic purposes in local contexts —which are far removed from the original evaluation process —will require substantial effort.1 6 To assist schools in interpreting the AWA scores, GMAC furnishes sample essays and a Scoring Guide designating the standards of achievement for each score (GMAC, nd, pp. 11-15; GMAC, 1994). Understanding the AWA scores will require synthesizing the standards for each score and reviewing the essay samples as the basis for determining what score constitutes an acceptable level of performance for the specific graduate program. Not only should this local cut-off score prove useful as an indicator of academic readiness for making admissions decisions, but it may provide a benchmark for making diagnostic decisions about placements and waivers for communications courses, as well as assignments to remedial work.1

GMAT Writing Assessment 20 However, making diagnostic decisions solely on the basis of AWA scores would be highly problematic because essays receiving any single score are not defective in the same respects. Low AWA scores only suggest that prospective students have one or more of several possible problems of some relative degree of severity. Thus, if schools want to use the AWA score as the sole screening device, they should be aware that all those individuals scoring below a certain level do not come with the same needs for further work in writing. Without further diagnosis, the only realistic alternative is to offer tutorials or very small classes designed and staffed to meet the individual needs of students with highly diverse writing deficiencies, including reasonably advanced writers with sophisticated problems in sentence coherence or logical organization; non-native speakers with language-specific learning needs; and otherwise effective writers whose texts are riddled with surface errors. If schools want to diagnose students' deficiencies and place them appropriately, or counsel students about their writing, the essays must be evaluated locally according to specific criteria that match the school's opportunities for further work in writing. Since local evaluators would be aware of the options and the appropriate placements for students exhibiting particular writing problems, such an in-house analysis of the AWA essays could be very effective. If schools decide to invest in such an effort, they would need to hire and train evaluators and establish a careful analytical evaluation process to identify essay features of significance for post-assessment consultations and placements. As some MBA programs have.already shown (Rogers, in press), such a local analysis could be used to furnish meaningful feedback to low-scoring students who would likely expect their writing problems to be specified. By furnishing the applicants' essays to the business schools, GMAC

GMAT Writing Assessment 21 wisely deferred diagnostic issues to the local personnel, who are in the best position to make placement decisions, and in so doing, minimized the costs to the test-takers. Ironically, however, by sending schools holistic scores and unmarked essays, GMAC may also have erected a potential barrier for many business schools, perhaps inhibiting use of the AWA for diagnostic purposes. The GMAT's value as a diagnostic instrument was perceived as "very useful" by GMAC survey respondents heading "mainstream" MBA programs; the survey results showed a significant difference between the elite programs that perceived the usefulness of the AWA to be for admissions, and the mainstream schools that perceived it to be for diagnostic purposes.1 8 The latter business schools, with less stiff entrance requirements, inevitably enroll some students with academic deficiencies, including some from disadvantaged backgrounds with critical writing problems. While such mainstream MBA programs might have considered the GMAT writing assessment to be an opportunity to help these students improve, their communication abilities may be the programs least likely to have the resources necessary to evaluate the essays and develop appropriate communication courses and remedial options. And without the money to do the job, these schools may simply sidestep the opportunity and admit low-scoring applicants who register for remedial work of some kind. In GMAC's many announcements and reports about the new writing test, there is a notable silence about the ramifications of using the AWA as a diagnostic tool and no indication that, without reviewing the essays, schools' diagnostic decisions would be fairly rough guesses. Nor does GMAC suggest the considerable costs individual MBA programs may incur if they intend to use AWA scores and essays for effective diagnostic purposes. The fee MBA applicants pay for the AWA goes to GMAC for scoring and processing the test. Analyzing the AWA scores and essays for diagnostic purposes is left to the

GMAT Writing Assessment 22 individual schools. For the mainstream schools that wanted the GMAT scores for placement purposes, and especially for those schools concerned about offering appropriate further work in writing for their students, the AWA results will provide little guidance. PREDICTING COMPETENCY IN MANAGEMENT WRITING Throughout the development and promotion of the AWA, GMAC was scrupulously careful to define what the test would measure and what its specific purposes would be. In GMAC's many official publications over the past couple of years, the AWA is always presented as a test of academic abilities for academic purposes. The AWA task is consistently described as "analytical writing" pertinent to "academic programs" in graduate schools of management and the AWA score is referred to as "an important indicator of academic performance" (GMAC, nd, p. 9). However, GMAC's 1993 survey reveals that business school respondents endorsed the AWA as a test of workplace as well as of academic writing ability. Analysis of the survey results reveals that among the many reasons survey respondents cited for incorporating the AWA into the GMAT, the most frequent response concerned the need for effective writing skills to succeed in business (Bruce, 1993, pp. 23 & 28). Summarizing respondents' support for the new test, GMAC announced that "effective writing skills are needed in order to succeed in both graduate school and business" (p. 2). GMAC classified respondents' workplace-writing endorsements as follows: Effective communication/writing skills are needed by managers and to succeed in business, career, and/or life. Complaints received from employers; pressure from business/alumni (Bruce, 1993, p. 28).

GMAT Writing Assessment 23 The prevalence of such survey responses suggests that expectations for the new test were not confined to academic performance, but also extended to the workplace. Although GMAC's survey showed many potential users assumed that the new test would be relevant to managerial writing competency for the workplace, GMAC never claims to fulfill this expectation. However, AWA announcements and promotional statements do not discourage inferences that the AWA is relevant to both the academy and the workplace, and some GMAC statements may lead some to associate the test with managerial writing. For example, the introductory brochure states that the AWA addresses "new management demands" in which the ability "to write effectively has become more important" (GMAC, nd, p. 1). Furthermore, several GMAC publications describe the AWA criteria as follows: In order to score well on both tasks, examinees must determine an effective communication strategy, present logical arguments, and express their ideas in language that is correct, concise, and persuasive. (GMAC, nd, p. 1). Announcing that the AWA will assess abilities to develop a "communication strategy," "present logical arguments," and to be "persuasive," may create an impression that the test covers rhetorical principles associated with management writing. Developing a communication strategy implies an audience and situation; being persuasive involves adapting content and language to achieve an objective with an audience (Shelby, 1 991); and presenting logical arguments implies developing claims, data, and warrants (Toulmin, 1958). In fact, the AWA Scoring Guide includes none of the elements traditionally associated with communication strategy. Rather, the AWA elevates self expression, with directions to "explain your opinion," recalling a literary notion almost antithetical to the workplace —a notion that

GMAT Writing Assessment 24 "you must write for yourself and not be gnawed by constant worry over whether the reader is tagging along behind" (Zinsser, 1990, p. 27). In contrast, reader or audience analysis is at the heart of communication strategy and persuasion in management communication. At a fundamental level, the GMAT writing assessment does not test writing for management because the prompts do not place the performer in a rhetorical context which requires him or her to analyze and adapt ideas and language for an intended audience. The AWA does not evaluate key attributes of management communication competency, such as the ability to interact with others to solve problems, to be sensitive to the needs and expectations of others, to represent the perspective of others and to negotiate between dissonant perspectives. The AWA does not assess the kind of functional discourse that MBA graduates will encounter in their business careers —writing which, above all, is transactional between writer and audience in a specific situation, accomplishing communication objectives in light of organizational considerations and constraints, objectives that are consequential for a variety of stakeholders (Cross, 1994; Driskill, 1989; Kleimann, 1993). Instead, the AWA evaluates the ability to present individual ideas and opinions as part of a logical analysis. It privileges academic discourse which is largely decontextualized. Despite implications that the AWA might be relevant to management writing, the test itself is true to GMAC's explicit and official descriptions: the AWA exclusively tests academic writing ability. Yet, might AWA results provide a reasonably accurate gauge of some writing abilities that are important in the workplace? Some of the AWA criteria (e.g., organization, syntax, diction, correctness) seem transferable, and certainly managers need to know how to state their opinions, and to analyze issues and arguments. Would it be

GMAT Writing Assessment 25 safe for deans, MBA program administrators, and even placement personnel, to conclude that those who score well on their AWA essays will write well on the job? Will the AWA identify those individuals who can be expected to write well in many management venues and situations? Not necessarily. Undoubtedly there are general cognitive abilities and textual skills which do transfer across writing types and situations, but expertise varies across disciplinary communities, as we noted above, "so there can be no one definition of an expert writer" (Faigley, 1986, p. 535). Each mode of writing demands its own set of abilities, and writers' specific abilities demonstrated on one type of task may not translate to other types of tasks (Purves, 1992). Although some writers will do well on any task assigned to them, others may score well on an academic task like the GMAT analytical essay, yet do poorly on a management writing task like a persuasive memorandum. Observing the best practices based on research, GMAC never directly claims that the AWA tests some undifferentiated kind of "good" writing, a kind of umbrella discourse that would embrace both school and workplace writing. However, GMAT users might assume that the AWA tests writing abilities for both school and work, as GMAC materials imply. Some users might infer that the AWA will distinguish "good" writers from "bad" writers, believing that the AWA measures writing abilities that are universal and transferable. If such assumptions are made, will it really matter? We believe it will. If, for example, the AWA is assumed to represent competence in managerial writing, scores may be used to signal students' expertise to business constituencies or to exempt students from courses where managerial writing is taught. Such decisions would be unwise, for while the AWA measures an important mode of academic writing and therefore may prove useful for admissions decisions, the

GMAT Writing Assessment 26 AWA scores and essays are not designed to predict applicants' competencies as managerial writers. Schools cannot assume that applicants who score high on the AWA do not need management communication training, nor can they assume that those scoring low necessarily need remedial work. Some testtakers may have difficulties with both academic and workplace writing; others may understand the complex conventions of school writing, but not of workplace tasks; still others may have sufficient competencies in both. Given the design of the AWA and what is currently known from writing and assessment research, schools should not expect that the AWA will predict applicants' competencies as managerial writers. At most, the AWA evaluates one important genre among an array in the academic community, a genre of limited relevance to the world of management. CONCLUSION Our analysis, summarized in Figure 2, indicates that the GMAT Analytical Writing Assessment stakes out a new position for writing in management education. It defines writing as a critical management activity, one that demands high-level cognitive abilities. In reviewing the new writing test, we have no doubt that it will provide business schools with useful information that is not currrently available: it will serve as an additional predictor of general academic success and it will provide schools with data on applicants' analytical writing abilities. However, the AWA only partially meets the expectations of management educators. Although the AWA uses writing effectively a a vehicle for measuring MBA applicants' general academic abilities, it is less effective as a test of their writing abilities relevant to management. As a test of analytical writing, the AWA is of only limited relevance either to MBA course writing or to managerial writing in the workplace. Rhetorical and social

GMAT Writing Assessment 27 aspects of writing are excluded, aspects that are fundamental to management. Furthermore, schools wishing to use AWA scores for diagnostic purposes must commit significant resources to interpret and use the test data, if they would take full advantage of the results. So despite all the publicity about a special writing test for the GMAT, the AWA appears to be a new packaging of a general academic assessment. [Insert Figure 2 About Here] The potential impact of the AWA and the concerns raised by its introduction place us at a critical juncture. As part of the long-established GMAT and as a test developed by ETS, the AWA could easily become a standard for writing competency in management graduate study and its results applied injudiciously. Its endorsement by deans and MBA program directors (whom GMAC surveyed) immediately suggested its perceived relevance to administrators in management education. Now, the AWA, a provocative addition to the GMAT, is becoming a topic of discussion among all the stakeholders in management education. Already, the AWA has motivated faculty and administrative dialogue about possible changes in MBA curricula and programs. New instructional designs, workshops, and courses may be envisioned to address needs identified by the AWA, and existing offerings may be modified or even eliminated based on assumptions about the test. MBA students are also contributing to the interaction about the AWA, and it is not farfetched to imagine a graduating MBA citing a high AWA score as a kind of "certification" of communication competency for prospective employers concerned about managers who can't write. The critical juncture finds managment communication scholar-teachers confronting a new test of communication abilities that is potentially useful, but one that has serious limitations. The AWA challenges communication specialists to become critically aware of the test's appropriate applications, and to exploit this

GMAT Writing Assessment 28 opportunity to develop writing assessment and writing instruction for management. Using the AWA as the impetus for confronting issues about the place of writing in MBA education, communications scholars can engage other constituencies in a dialogue to facilitate both truly appropriate application of the test and the development of a cutting-edge writing assessment instrument that is directly relevant to management. Conversation among all the constituencies of the management education community —deans and administrators, faculty, MBA students, business, and other organizations hiring MBA's —is essential to develop full awareness of the test's opportunities and limitations so that schools can appropriately incorporate the AWA into the educational experience of management students. For example, even a simple thing like using the AWA essays as classroom examples, merits forethought since the essays do not represent a complete picture of writing for MBA studies and particularly since they are markedly different from the kind of writing taught in business and management communication courses. Exploring these differences in light of national and local educational goals, student needs, and business requirements could be advantageous for all management education's constituencies. Larger programmatic changes in response to the AWA should be regarded with even greater caution. Some schools, for example, may consider the AWA as a replacement for local MBA writing assessments. However, schools should not assume that the AWA fulfills the same objectives as MBA assessments that feature management case prompts. Through these case prompts involving elaborated business scenarios, MBA assessments test the rhetorical and social aspects of writing pertinent to some writing practiced in management courses and to discourse taught in management communication classes. Most significant, the writing tasks in MBA assessments are relevant to managerial writing in the workplace.

GMAT Writing Assessment 29 At some point in the development of the AWA, GMAC decided against using a case prompt that assigns applicants to compose a management mode of writing (Hecht and Alloway, 1984). Instead, the AWA was developed around a traditional academic instrument, analytical writing —a general academic writing task that may be the most feasible assessment mode for the diverse population of GMAT testtakers whose undergraduate majors and business experience vary widely. But this choice of an analytical writing task, as we have argued, reduces the relevance of the AWA for business schools. In addition to facilitating the application of the AWA results with full awareness of the test's limitations, management communication specialists should also become advocates for state-of-the-art management writing assessment. The GMAT essay test, like all large-scale national writing assessments, is based on what many consider to be an outmoded assessment model —the timed, writingsample approach that constrains a writer's composing process, truncating it to first-draft writing and isolating it from its social and cultural context. Theory and research on writing, both from cognitive-processing and social perspectives, critiques the writing-sample model of assessing writing, finding it does not represent any of the current theoretical constructs for writing (Camp, 1993; Purves, 1992; Witte, 1992). Other approaches to writing assessment for management development, such as MBA assessments, make an effort to capture the social dimension, but that effort is only partially successful (for example, writers responding to a case can't interact with a manager in planning a document), and these local tests also fail to allow writers the opportunity to engage in the full naturalistic writing process. Newer assessment models are superseding timed writing-sample tests for freshman-level placement at many universities (Miami University, University of Minnesota); most notably, "portfolio" assessments collect several types of writing over a period of time; allow revision,

GMAT Writing Assessment 30 contemplation, and feedback; and encourage contextual commentary and interpretation by the writer (Belanoff & Dickson, 1991; White, 1994). Though not a ready alternative for such large-scale tests as the GMAT, portfolio assessments suggest new possibilities to more adequately test management writing. The AWA is welcomed, particularly for the attention it brings to writing as a significant management ability requiring more than superficial knowledge of language skills. However, if MBA students are to be equipped to write successfully in MBA programs and in the workplace, writing assessment and instruction in MBA programs must exceed the bounds of the GMAT Analytical Writing Assessment.

Analysis of an Issue Analysis of an Argument People often complain that products are not made to last. They feel that making products that wear out fairly quickly wastes both natural and human resources. What they fail to see, however, is that such manufacturing practices keep costs down for the consumer and stimulate demand. Which do you find more compelling, the complaint about products that do not last or the response to it? Explain your position, using relevant reasons and/or examples drawn from your own experience, observations, or reading The following appeared as part of an article in a weekly news magazine: The computerized, on-board warning system that will be installed in commercial airliners will virtually solve the problem of midair plane collisions. One plane's warning system can receive signals from another's transponder —a radio set that signals a plane's course — in order to determine the likelihood of a collision and recommend evasive action. Discuss how logically convincing you find this argument. In explaining your point of view, be sure to analyze the line of reasoning and the use of evidence in the argument. Also discuss what, if anything, would make the argument more sound and persuasive or would help you to better evaluate its conclusion. Figure 1: Sample Prompts for the GMAT Writing Test (GMAC, nd, p. 3)

Contributions Limitations Tests Writing Performance as High-Level Excludes Rhetorical/Social Aspects of Ability Writing Tests Analytic Writing Represents Only One Type of Writing for MBA Study Provides Scores Indicating Academic Ability Identifies No Particular Features to Account for the Scores Furnishes Written Essays and Scored Samples Provides No Specific Diagnostic Information Recognizes Writing as a Critical Management Does Not Test Management Writing Activity Figure 2: Contributions and Limitations of the GMAT Analytic Writing Assessment

GMAT Writing Assessment 31 NOTES 1. The total GMAT score (combining the Verbal and Quantitative portions, each of which is also reported separately) excludes the AWA score and is reported to schools in the same fashion as the old GMAT. Thus, faculty and administration of each business school must determine the specific purposes, if any, for which they will use the writing test scores. The independent reporting of the AWA score permits individual schools to ignore the writing test, if they wish, at least temporarily. 2. Subjectivity of raters in evaluating papers still concerns 4 percent of GMAT-subscribing schools (Bruce, 1993, p. 21). Also, see Bruce, 1992. 3. To accommodate the AWA, the quantitative and verbal portions of the GMAT were each reduced by 15 minutes, and the total GMAT time was extended by 30 minutes for a total of 4 hours and 30 minutes. Test design, administration, and scoring of the AWA add a fee, originally set at $17, to the cost of the GMAT (Broesamle, 1993). 4. Over the past decade and more, but particularly since the publication of such milestones as Boyatzis's The Competent Manager (1982) and Porter and McKibben's (1988) study of management education, there has been a highly visible movement to "reinvent the MBA," including the addition of training programs for interpersonal skills, team-building, and leadership through "Outward-Bound" type activities and practical assignments (Bigelow, 1991; Economist, 1991; Fortune, 1994). These programs are based on a growing belief that strong verbal abilities are of greater importance than narrow technical skills for effective management, a belief exemplified by the fact that the most common ability sought by organizations hiring MBA's today is communication ability, both in written and oral form (O'Dowd & Liedtka, 1994). 5. The GMAC Survey of attitudes toward a writing assessment was conducted in March 1993. The survey covered all GMAT-using institutions; the final sample included 355 schools (65 percent), with slightly disproportionate representation of GMAC members and better schools with fulltime programs. Respondents were primarily deans and MBA program administrators. 6. Although some GMAC survey respondents expressed concern that an essay test might be discriminatory to some MBA applicants (Bruce, 1993), research suggests that it is the indirect language skills tests that tend to discriminate toward some minority groups, probably because of the emphasis on conventions and usage (White, 1994). 7. Post-secondary, national performance assessments of writing include professional school tests, such as the Medical College Admission Test (MCAT) and the Law School Admission Test (LSAT), and general academic assessments, such as the Scholastic Assessment Test (SAT, formerly the Scholastic

GMAT Writing Assessment 32 Aptitude Test) and soon the Graduate Record Examination (GRE). The Educational Testing Service (ETS), a major player in the development of national writing tests as well as the developer of the original GMAT to be a specialized test of the abilities for graduate study of business, also collaborated with GMAC on developing the AWA (Snook, 1987). 8. Despite the wide acclaim for holistic scoring methods, they have not gone unchallenged, especially in terms of the procedures necessary to achieve reliability (such as imposing constraints on the natural reading process, constraints which may achieve reliability at the expense of validity (Huot, 1990). Certainly some of the comments in the GMAC reports tend to create an overly optimistic sense of technical perfection in holistic scoring; for example, discrepant scores are hardly "rare" and consistency from one test administration to the next is not a foregone conclusion (GMAC, nd, pp. 6-7; see Cherry & Meyer, 1993). 9. Local performance assessments of the writing of future managers exist not only in MBA assessment programs, but also in the workplace in Assessment Center programs, and each has something significant to offer management education that is not available in the GMAT writing test. However, because both local MBA assessments and workplace Assessment Centers are costly and labor intensive, neither has been widely adopted, and neither has influenced, in any way, the development of the GMAT writing test, a test which is a direct descendant of other ETS-sponsored large-scale assessments. 10. Schools with mean GMAT scores over 500 tended to see some usefulness in the AWA for admissions purposes, but it was primarily elite schools with mean GMAT scores above 600 that saw admissions as the primary or even sole use for the AWA (Bruce, 1993, pp. 11-13). (See Bruce p. 34 for definitions of the top-level schools.) 11. The two essays of the same genre or type of writing on different topics should improve the validity of the test (Brown, Hilgers, & Marsella, 1991). Although packaged differently and accompanied by separate scoring guides, the two prompts clearly represent the same genre, with the advice for test-takers appearing in a single list for the two tasks: "[The evaluators] will consider how well you organize, develop, and express your ideas about the issues or argument; provide relevant supporting reasons and examples, and control the elements of standard written English" (Kelly, McHale, Munter, & Zak, 1993). 12. In addition to the criteria listed, the Scoring Guide directs evaluators to look at language use and control of textual conventions. 13. GMAC explicitly contrasts the AWA with expository writing (Bruce, 1993, p. 8). 14. The backgrounds of the raters are a significant factor informing the holistic scores. Because all raters tend to be influenced by their own experience as well as the mandated criteria, those raters of similar experience most readily achieve consensus (Pula & Huot, 1993). Studies have

GMAT Writing Assessment 33 demonstrated the external validity of specially trained "professional" judges, however, showing that their ratings do generalize to others' perceptions of writing quality (Bochner, Albertini, Samar, & Metz, 1992). 15. The AWA score is the average of four holistic ratings assigned by four different evaluators. At least two evaluators independently score each essay; if their scores differ significantly, other evaluators rate the paper until adequate agreement is achieved. Individual scores do not include outliers, though reliability rates are calculated on all ratings (see Cherry & Meyer, 1993). 16. National data on the AWA should become available after a full year of administering the new GMAT. After some experience with the AWA, perhaps GMAC will report combined Verbal and AWA scores, as well as the individual test scores; some research shows that a combined score is more valid than direct writing tests alone (White, 1994). 17. GMAC advises that schools should eventually conduct a local validity test of the AWA (under GMAC's auspicies) to determine if the scores they have selected reflect their actual needs (GMAC, nd; see Stolzenberg & Relles, 1985). 18. Schools with GMAT scores under 600 (that is, all but the top schools) tended to consider the AWA useful for diagnostic purposes. The correlation between the two usefulness questions is.59, suggesting different needs are perceived by different types of schools (Bruce, pp. 11-13).

REFERENCES Ackerman, J. (1991). Reading, writing, and knowing: The role of disciplinary knowledge in comprehension and composing. Research in the Teaching of English, 25, 133-178. Ackerman, J. (1993). The promise of writing to learn. Written Communication, 10, 334-370. American Assembly of the Collegiate Schools of Business. (1993, April). Achieving quality and continuous improvement through self-evaluation and peer review: Standards for accreditation - business administration and accounting. St. Louis: AACSB. Applebee, A. N., Langer, J. A., Mullis, I. V.S., Latham, A. S., & Gentile, C. A. (1994). NAEP 1992 report card. Washington: U.S. Department of Education. Belanoff, P., & Dickson, M. (Eds.) (1991). Portfolios: Process and product. Portsmouth, NH: Heinemann Boynton/Cook. Bigelow, J. D. (1 991). Managing skills: Explorations in practical knowledge. New York: Sage Publications. Bochner, J.H., Albertini, J.A., Samar, V.J., & Metz, D.E. (1992) External and diagnostic validity of the NTID writing test: An investigation using direct magnitude estimation and principal components analysis. Research in the Teaching of English, 26, 299-313. Boyatzis, R.E. (1 992). The competent manager. New York: John Wiley. Broesamle, W. ( 1 993, September). GMAT analytical writing assessment. (Available from Graduate Management Admission Council, 2400 Broadway, Suite 2030, Santa Monica, CA 90404.) Brown, J.D., Hilgers, T., Marsella, J. (1991). Essay prompts and topics: Minimizing the effect of mean differences. Written Communication, 8, 533-556. Bruce, G. D. (1 992). Exploratory marketing research on the addition of an analytical writing assessment to the GMA T. (Available from Graduate Management Admission Council, 2400 Broadway, Suite 2030, Santa Monica, CA 90404.) Bruce, G. D. (1 993). Attitudes toward the addition of an analytical writing assessment to the GMA T: Results of a survey of institutions using the GMAT. (Available from Graduate Management Admission Council, 2400 Broadway, Suite 2030, Santa Monica, CA 90404.) Camp, R. (1993). Changing the model for the direct assessment of writing. In M. Williamson and B. Huot (Eds.), Validating holistic scoring for writing assessment: Theoretical and empirical foundations (pp. 45-78). Cresskill, NJ: Hampton Press.

Carter, M. (1990). The idea of expertise: An exploration of cognitive and social dimensions of writing. College Composition and Communication, 41, 265-286. Cherry, R.D., & Meyer, P.R. (1993). Reliability issues in holistic assessment. In M. Williamson and B. Huot (Eds.), Validating holistic scoring for writing assessment: Theoretical and empirical foundations (pp. 109-141). Cresskill, NJ: Hampton Press. Cross, G.A. (1994). Collaboration and conflict: A contextual exploration of group writing and positive emphasis. Cresskill, NJ: Hampton Press. Devitt, A.J. (1991). Intertextuality in tax accounting: Generic, referential, and functional. In C. Bazerman & J. Paradis (Eds.), Textual dynamics of the professions: Historical and contemporary studies of writing in professional communities (pp. 336-57). Madison: University of Wisconsin Press. Driskill, L. (1989). Understanding the writing context in organizations. In M. Kogen (Ed.), Writing in the business professions (pp. 125-145). Urbana, IL: National Council of Teachers of English & Association for Business Communication. Driskill, L., with Ferrill, J., & Steffey, M. N. (1992). Business & managerial communication: New perspectives. Fort Worth, TX: Dryden Press. Durst, R. K. (1987). Cognitive and linguistic demands of analytic writing. Research in the Teaching of English, 21, 347-376. Economist, The. (1991, March 2). A survey of management education: Passport to prosperity, 3-26. Engelhard, Jr., G., Gordon, B., & Gabrielson, S. (1991). The influences of mode of discourse, experiential demand, and gender on the quality of student writing. Research in the Teaching of English, 26, 299-314. Faigley, L. (1986). Competing theories of process: A critique and a proposal. College English, 48, 527-42. Farr, M. (1993). Essayist literacy and other verbal performances. Written Communication, 10, 4-38. Forman, J. (1989). The discourse communities and group writing practices of management students. In C. Matalene (Ed.), Worlds of writing (pp. 247-254). New York: Random House. Forman, J., & Rymer, J. (1993, October). Academic genre in business: Exploring the discrepancy with the workplace. Paper presented at the meeting of the Association for Business Communication, Montreal. Freedman, A., Adam, C., & Smart, G. (1994). Wearing suits to class: Simulating genres and simulations as genre. Written Communication, 11, 193-226.

GMAC (Graduate Management Admission Council). (1990). Leadership for a changing world: The future role of graduate management education. Santa Monica, CA: Author. GMAC (Graduate Management Admission Council). (nd, distributed 1994, May). The GMA T analytical writing assessment: An introduction. Santa Monica, CA: Author. GMAC (Graduate Mangement Admission Council). (1994 forthcoming). Official guide for GMAT review. Santa Monica, CA: Author. Hecht, L. W. & Alloway, S. S. (1984). Developmental and operational plans for the graduate management writing assessment. Princeton, NJ: Graduate Management Admission Council & Educational Testing Service. Huot, B. (1990). Reliability, validity, and holistic scoring: What we know and what we need to know. College Composition and Communication, 41, 201-211. Jolliffe, D. A. (1988). Writing in academic disciplines. Norwood, NJ: Ablex. Kelly, C., McHale, F., Munter, M., & Zak, M. (1994, April). Analytical writing assessment. Paper presented at the meeting of the Management Communication Conference, New York. Kleimann, S. (1993). The reciprocal relationship of workplace culture and review. In R. Spilka (Ed.), Writing in the workplace: New research perspectives (pp. 56-70). Carbondale: Southern Illinois University Press. Koenig, J. A., & Mitchell, K. J. (1988). An interim report on the MCAT essay pilot project. Journal of Medical Education, 63, 21-29. McCloskey, D.N. (1985). The rhetoric of economics. Madison: University of Wisconsin Press. Mclsaac, C.M., & Aschauer, M.A. (1990). Proposal writing at Atherton Joran, Inc. Management Communication Quarterly, 3, 527-560. Moore, C. M. (1994). In L. Bickman & D. J. Rog (Eds.), Group techniques for idea building: Applied social research methods series, 9, Thousand Oaks, CA: Sage. Munter, M. (1 989a). Emerging trends in management communication. St. Louis, MO: American Assembly of Collegiate Schools of Business. Munter, M. (1989b). What's going on in business and management communication courses. In M. Kogen (Ed.), Writing in the business professions (pp. 267-278). Urbana, IL: National Council of Teachers of English & Association for Business Communication. Munter, M. (1990). Management communication. Vol 14 in Business administration reading lists and course outlines. Durham, NC: Eno River Press.

O'Dowd, K. 0., & Liedtka, J. (1994, Winter). What corporations seek in MBA hires: A survey. Selections: The Magazine of the Graduate Management Admission Council, 34-39. Olsen, L. A. (1993). Research on discourse communities: An overview. In R. Spilka (Ed.), Writing in the workplace: New research perspectives (pp. 181-194). Carbondale: Southern Illinois University Press. O'Reilly, B. (1994, January 24). Reengineering The MBA, Fortune Magazine, 38-45. Penrose, A. M. (1992). To write or not to write: Effects of task and task interpretation on learning through writing. Written Communication, 9, 465-500. Porter, L. W. & McKibben, L. E. (1988). Management education and development: Drift or thrust into the 21st century. New York: McGrawHill. Pula, J. J., & Huot, B. A. (1993). A model of background influences on holistic raters. In M. M. Williamson & B. A. Huot (Eds.) Validating holistic scoring for writing assessment: Theoretical and empirical foundations (pp. 237-265). Cresskill, NJ: Hampton Press. Purves, A. C. (1992). Reflections on research and assessment in written composition. Research in the Teaching of English, 26, 108-122. Quellmalz, E.S., Capell, F.J., & Chou, Chih-Ping. (1982). Effects of discourse and response mode on the measurement of writing competence. Journal of Educational Measurement, 19, 241-258. Rogers, P. S. (in press). Analytic measures for evaluating managerial writing. Journal of Business and Technical Communication. Rogers, P. S., & Horton, M. S. (1992). Exploring the value of face-to-face collaborative writing. In J. Forman (Ed.), New Visions of Collaborative Writing (pp. 120-146). Portsmouth, NH: Boynton/Cook Publishers. Russell, D.R. (1991). Writing in the academic disciplines, 1870-1990. Carbondale: Southern Illinois University Press. Ruth, L., & Murphy, S. (1988). Designing writing tasks for the assessment of writing. Norwood, NJ: Ablex. Scholtes, P. R. (1988). The team handbook: How to use teams to improve quality, Madison, WI: Joiner Associates Inc. Schumacher, G. M., & Nash, J. G. (1991). Conceptualizing and measuring knowledge change due to writing. Research in the Teaching of English, 25, 67-96. Schwindt, R. (1990). Organizational behavior. Vol 7 in Business administration reading lists and course outlines. Durham, NC: Eno River Press, Inc.

Snook, J. L. (1987). The history of GMAC. Los Angeles, CA: Graduate Management Admission Council. Shelby, A. N. (1 991). Applying the strategic choice model to motivational appeals: A theoretical approach. Journal of Business Communication, 28, 3, 187-212. Shelby, A. N. (1994). Selected MBA management communication programs. Unpublished manuscript. Georgetown University School of Business, Washington. Smeltzer, L. R., & Thomas, G. F. (1994). Managers as writers: A metaanalysis of research in context. Journal of Business and Technical Communication, 8, 186-211. Stolzenberg, R.M. & Relies, D.A. (1985). Calculation and practical application of GMAT predictive validity measures. Los Angeles, CA: Graduate Management Admission Council. Toulmin, S. (1958). The uses of argument. Cambridge: Cambridge University Press. Webster, L., & Ammon, P. (1994). Linking written language to cognitive development: Reading, writing, and concrete operations. Research in the Teaching of English, 28, 89-109. White, E. M. (1993). Holistic scoring: Past triumphs, future challenges. In M. M. Williamson & B. A. Huot (Eds.) Validating holistic scoring for writing assessment: Theoretical and empirical foundations (pp. 79-1 08). Cresskill, NJ: Hampton Press. White, E. M. (1 994). Teaching and assessing writing. (2nd. ed.) San Francisco: Jossey-Bass. Williamson, M. M. (1993). An introduction to holistic scoring: The social, historical and theoretical context for writing assessment. In M. M. Williamson & B. A. Huot (Eds.) Validating holistic scoring for writing assessment: Theoretical and empirical foundations (pp. 1-43). Cresskill, NJ: Hampton Press. Witte, S. (1992). Context, text, intertext: Toward a constructivist semiotic of writing. Written Communication, 9, 237-308. Zinsser, W. (1988). On writing well. (3rd. ed.) New York: Harper & Row.