THE UNIVERSITY OF MICHIGAN COMPUTING RESEARCH LABORATORY1 MERIT'S EVOLUTION - STATISTICALLY SPEAKING Eric M. Aupperle CRL-TR-4-84 JANUARY 1984 Room 1079, East Engineering Building Ann Arbor, Michigan 48109 USA Tel: (313) 763-8000 1Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the funding agency. Thf's a^;f a&in aippeiry —ed In EEE ToansaictioRs o Cow mpu^trs, Oi.C-32,NOaO10 (October), 1983.

IEEE TRANSACTIONS ON COMPUTERS, VOL. C-32, NO. 10, OCTOBER 1983 881 Merit's Evolution-Statistically Speaking ERIC M. AUPPERLE, SENIOR MEMBER, IEEE Abstrct-Merit links four Michigan universities with a packet- time-sharing and batch-processing systems. The communiswitcig comrter network. This network provides several interactive cation and processing features of these systems are portrayed and batch services to the students, faculty, and researcbers of these schools. While the network and its services are reviewed, this paper n Fig 1 focuses on the statistical performance of Merit during its first ten MSU: The Michigan State University (MSU) host comoperational years. Numerous graphs and tables present data including puter is a Control Data Corporation (CDC) Cyber 170 Series time-series showing the growth of usage, transmission-rate related 750 with 131 000 60-bit words of central memory and 500 000 measures, normalized measures on a per-connection basis, and the words of extended core storage. It runs under a locally-modidistribution of activity among the hosts. The data reveal that on an aggregated basis the current average packet size is nearly 50 bytes withb id version of CDCs NOS/BE operating system and supports a corresponding transmission rate of 1350 packets per hour for each nine remote-batch stations and 110 terminal ports in addition virtual connection. While these are the aggregate averages, there is to its network ports. MSU also has a CDC 6500 with 98 000 considerable variation of these values among the different services words of central memory that does not have a direct connection Merit offers. The paper concludes with explanations for these dif- to the network. ferences and general observations about Merit's usage patterns. M Michigan State's CDC hardware features a 60-bit word, Index Terms-Data communications, interuniversity networking, in contrast to the 32-bit word of the Amdahl hardware at the network statistics, packet switching, regional network, resource University of Michigan and WSU and to WMU's 36-bit word sharing. on their DEC system. Thus, one of MSU's principal contri~I. INTRODUCTION.butions to the network is as a supplier of extended-precision numerical and scientific computing. MSU also offers the CDC T HE Merit Computer Network is a packet-switching program-product versions of such languages as Fortran, Basic, network developed by three Michigan universities: and APL as well as CDC versions of SPSS and Pascal, all of Michigan State University, the University of Michigan, and which feature extensions to the corresponding implementations Wayne State University. The network became operational in of these languages at the other hosts. July 1972 and has operated continuously since then. UM: The University of Michigan (UM) has an Amdahl Throughout this period, the network's functional facilities have 5860 with sixteen million bytes of real memory running under been enhanced and its performance refined. In January 1979 a locally-written virtual-memory operating system, MTS (the Western Michigan University became the fourth Merit Michigan Terminal System). The UM host supports twelve member. remote batch stations and approximately 800 interactive terThe first section of this paper offers an overview of the minal ports. network's current characteristics and capabilities. This The Amdahl 5860 provides a fast and powerful network host background information provides the context for under- that is also-like the WSU system-able to manipulate very standing the main topic of the paper, the network's usage large data bases. Packages such as MIDAS, an interactive statistics. The next section describes statistical information statistical system, that have been written at UM to take adthat have been recorded throughout the network's operational vantage of this combination, account for a good portion of life and how these data are processed. This is followed by network usage at UM. several sections that present various statistical summaries and Two other UM hosts are an IBM System 370/148 used for data represented in graphical and tabular forms. In the final an experimental campus-wide word-processing support service section the author offers several observations about these and a DEC VAX 11/780 running UNIX used for image network data. processing and robotics research and education. These computing systems and a third, a recently added Prime 750, are II. AN OVERVIEW OF THE MERIT COMPUTER primarily used for research and their usage is not reflected in NETWORK data presented in this paper. A. The System WMU: The host computer at Western Michigan University (WMU) is a DECsystem 1099 with two KL10 processors and The network connects eight separate university computers 512K 36-bit words of main storage. It runs DEC's TOPS-10 in southern Michigan, most of which are large combined perating system, the only ho on the network to offer interactive service on an unmodified vendor system. Manuscript?eived January 29, 1980; revised March 28, 1983.rvi o u ii ste The author is ittht Merit Computer Network, 2200 Bonisteel Blvd., Ann WSU: The principal host computer at Wayne State UniArbor, MI 48 l i:;. versity (WSU), which also runs MTS, is an Amdahl 470V/6 0018-9340/83/1000-0881S01.00 C 1983 IEEE

882 -! |!i |e i Ii Ie r s(1 M sll^Il, Lp;b! "' ii! /SJE Ii KE I / \ Ij / / \ S / Ii'I: ~\li lS iii1 I *.XA* U bI I~i]H V"ill 1 ii i i~U f E

AUPPERLE: MBRIS EVOLUTION 883 with eight million bytes of solid-state memory. The system computers for transmission on the network; to receive messages supports 282 interactive terminal ports and five remote batch from other nodes for delivery to their hosts or for forwarding stations. The WSU Computing Services Center emphasizes to another host; and to support local dial-up terminals. the production of high quality printed copy for use in data The PDP-11 of each CC is augmented with several intertables, graphs, and even the full texts of technical reports. faces as shown in Fig. 2. The major ones are the host and the A significant and unique network resource at WSU is IBM's asynchronous and synchronous communications interfaces. MVS, which runs on a second Amdahl 470V/6 and serves their The synchronous communications interfaces are designed to administrative data processing needs. While MVS is not yet transfer entire messages over the telephone network directly accessible interactively through the Merit network, it does to and from the PDP- l's memory without program interprovide batch service to network users. This makes available vention, once the transfer is initialized by software action. Each a number of program products written for IBM's OS or MVS one is controlled by its own microcomputer. Currently, two or that are not available at the other hosts. more of these synchronous ports are used at each node. Augmenting these host systems is the Merit Network. The The synchronous interfaces are connected to AT&T Model technological design of the Merit Computer Network employs 208A Data Sets except for the Ann Arbor-Detroit link, which communications switching computers which act both as in- uses Dataphone Digital Service, and the internal UM links terfaces between the host computers and the network and as which use limited-distance modems or direct connections. The packet-switching nodes. One or more Communications data transmission is full-duplex at rates from 4800 bits/s to Computers (CC) is located at each host site. The six CC's are 57.6 kbits/s as shown in Fig. 1. The interfaces are designed to interconnected by means of modems through leased telephone operate at rates in excess of 500 kbits/s, however, when lines as shown in Fig. 1. The interfaces that the CC's present equipped with higher data-rate modems. Each asynchronous to the host systems are uniquely adapted to the requirements interface provides for up to 32 local dial-up ports. Each port of each particular host; the interfaces that the CC's present is equipped with either an auto-baud detect modem for operto the network (and thus to each other) are identical. ation up to 300 bits/s or with a 1200 bits/s modem for operOne of the CC's is a Digital Equipment Corporation ation at that data rate. PDP- 11/20's with 28K 16-bit words of memory. Four others, All but one of the host interfaces are special hardware dethe ones on the UM and WSU campuses, are PDP- 11/60's, vices that allow for the parallel transmission of data records each with at least 32K words of memory. The sixth, at WMU, to and from the memory of the associated CC's, performing is a PDP- 1 /40 also with 32K words of memory. The primary whatever memory alignment operations are required for word functions of the Communications Computers are to multiplex lengths which differ from that of the CC. The multiple-address virtual connections over physical communications links host interfaces simplify the logical structure of multiplexing (communicating with other nodes using the HDLC link-level user activity over the network and thereby reduce the amount protocol); to receive messages from their respective host of network support software required in the host computers. Full Duplex Dial Up Lines intrvl 2 Th c ca haynchronoua Timor Asynchrono No tI-r ==in - -232 */Multiplno-r' / Lb —Ior Ifr to 32) i / -' k Hoat Comp1-r a we For-IIymahronou I -- erwororeooor. -*23 w... -- Multiplexor!'-t mi~~~~~~~~~~rll DuIplex t —imal Dedicated Linm8 Fig. 2. The oommunications computer hardware.

884 IEEE TRANSACTIONS ON COMPUTERS, VOL C-32, NO. 10, OCTOBER 1983 I I _..I * i;i # l m' 1. * m 4 x Fig. 3. Number of different network users per month. Additional hardware details appear in [1] and [4]. The Prime type of service obtained from the hosts, and the type of host 750 host is interfaced to the network through a serial X.25 access employed by the user. communications link. There are four types of network access: host-to-host interIt must be emphasized that the Merit network augments the active, direct terminal, external, and network batch. Brief existing terminal and batch access facilities provided by each descriptions of each follow. host system. The data reported in subsequent sections relate Host-to-Host Interactive Access: This service requires that only to traffic on the network rather than to the total access a user (or a user program) establish a connection from a local activity provided by the hosts through their local front-end host to a selected remote host. The resources at the remote'ost systems. that are made available by the network connection are the same as those available through a terminal directly attached B. Types of Network Service to that host. The user (or a user program) may now use these The network offers users of any one computing center access resources and does so by issuing commands, etc., in the manner to the facilities of the others. To provide this access the con- appropriate to the remote host. In general, however, the user puters at each center, called hosts, are interconnected through is able to use the terminal in the manner usual to the local host. the network. The user communicates information between This service has four subtypes. hosts through a virtual connection, more simply called a Classic Interactive: the interaction possible among two connection. This may be an information path from one host or more of the network hosts without the expense of normal to another dedicated to the user. It may also be from Hermes, long-distance telephone transmission. The network reduces the network's terminal access facility, to a serving host. The the rate of communication error and makes it possible for users initiating end of a connection is the local host and the other to work with the local terminal operating conventions and end is a remote host. The exact appearance of an interhost keyboard editing modes to which they are accustomed. connection depends upon the local host's operating system, the Enhanced Interactive: which means simply that data

AUPPERLE: MERIS EVOLUTION 885 IX, II.. i. Y Y w TO _JM 1 Fig. 4. Number of successful network connections. bases from one of the host computers can be used with an ex- users to concurrently access any of the Merit hosts through ecuting program at another host, or on more than one addi- GTE Telenet. tional host. It is also possible to display or print out data from Network Batch Access: This setvice allows a user to submit a network task at any of the hosts. an execute, print, or punch job at any of the network's hosts. File Transfer: which allows data to be copied from one The job may be, and usually is, initiated from a host other than host to another, whether from disk to disk, disk to tape, disk the one where it is to be processed. Establishing the connections to printer, or in any of several other combinations. for transmitting the job and for retrieving any output is done Interprocess Communication: the process of running by the Network Batch Service. programs on one or more host computers concurrently; pro- Remote Job Entry. allows any batch job to be submitted grams may be employed simultaneously or sequentially, and either in the form of a card deck or from a terminal. The systhey may transfer data from one host to another. tem where the job is to be processed need not be operating at Ditrect Terminal Access: This is the capability for dialing the time. Network Batch Service will arrange for the job's directly into the network from a terminal and accessing a re- transmission when the requisite host computers are opermote host without going through a local host. This facility is ating. provided by Hermes, Merit's network-to-terminal interface, Batch File Transfer: allows data to be copied from one which runs in each of the individual Communications Corn- host to another, e.g., from disk to disk, disk to tape, or disk to puters. printer in batch mode. This form of file transfer is often External Access: This is implemented through an X.25 link cheaper and faster than interactive file copying. operating at 4800 bits/s between the GTE Telenetj network and the Merit Networkl. This service currently allows up to 32 111. A DESCRIPTION OF THE NETWORK'S STATISTICAL LOGGING AND PROCESSING PROCEDURES! "GTE Telenet" is a relistered scrvie mark of OTE Tlenet Communications Corporation. Each time a virtual connection either enters or leaves a

886 IEEE TRANSACTIONS ON COMPUTERS. VOL. C-32. NO. 10. OCTOBER 1983 II Fig. 5. Elapsed connection time (hours). serving host, data about this connection are accumulated by tion type. Network connection types are associated with the the host computer. When a connection finally terminates, the various kinds of network services. Those types relevant to this connection's statistics are logged by the host. The recorded report are: information includes the following: * Host-to-Host Interactive Access-IT * date and beginning time of connection * Direct Terminal Access (Hermes)-TL * duration of connection * External Access (Merit-Telenet)-X3 * connection's initiating and receiving hosts * Network Batch Access-EX and PR * connection type The IT, TL, and X3 types represent interactive connections. * a code indicating whether the connection was suc- They differ in that an IT connection is between a pair of serving cessfully opened or the reason for its failure hosts while the TL and X3 types are between a terminal and * both the number of packets and the number of bytes a host. The Hermes and Merit-Telenet internetwork service sent and received during the existence of the connection connections are further distinguishable by their initiating e user's account ID. pscudohost2 ID's. For batch service, the network differentiates These data are continuously accumulated in on-line host between connections carrying jobs for remote execution, EX, files. Once each month the contents of these files are stored on and connections transmitting jobs for printing, PR. tape and the files emptied. The monthly aggregates from all One of the statistical programs run monthly separates the hosts are combined and processed by a comprehensive set of aggregate data records into two groups: records produced by statistical analysis programs. The results of these analysis a connection leaving a host, called Outbound, and records programs are used to prepare various network usage level reprogrts. are used to prepae various networ usage level re- 2 Each Hermes site and the Merit-Telenet interconnection are treated as hosts by the network. Hence, each has its own host ID just as the computing Most of the statistical record parameters listed above are center hosts do. The modifier "pseudo" is used to identify noncomputing-center self-explanatory. An important one that is not, is the connec- hosts.

AUPPERLE: MERITS EVOLUTION 887 i * 6 U S * 0" &M -N A W Sw'ai0 ", " " Fig. 6. Number of kilopackets transmitted. produced by a connection coming into a host, the Inbound Number of Network Users-the number of different connections. These groups are processed separately but in es- computing ID's successfully opening connections each sentially the same manner. The results produced for each group month. include: Successful Network Connections-the total number of * A listing of the individual user ID's. successfully-opened connections each month. * Statistical measures for connect time and transmitted Elapsed Connection Time-the total number of hours bytes and packets for each connection type. during which all successfully-opened connections remained * An analysis of the distribution of connections between connected duringeach month. all pairs of network hosts. Transmitted Packets-the total number of packets transFor outbound connections, an analysis of the failure mitted over all successfully-opened connections each month, modes. expressed in kilopackets. Each packet may contain up to 240 Only a small portion of the available information produced bytes of user data, but is no longer than necessary. in these monthly summaries is presented in this paper. Transmitted Characters -the total number of characters, as eight-bit bytes, transmitted over all successfully-opened connections each month, expressed in megabytes. IV. GROWTH OF THE NETWORK Plots depicting each of these time-series are shown in Figs. 3-7. The horizontal axis in each plot represents a ten-year span One purpose of the network's statistics is to record and re- beginning in July 1972. Grid or tick marks appear at six-month port its use with time. This is done by maintaining a set of intervals. The scale on the vertical axis varies from one plot to time-series, based on the monthly summaries, that describe another. In each plot the solid curve presents Inbound data total network traffic. Five of these time-series are as fol- while the dashed line represents a seventh-degree polynomial lows: curve fitted through the respective data points.

888 IEEE TRANSACTIONS ON COMPUTERS. VOL. C-32, NO. 10. OCTOBER 1983 i0~~ di 72\,^~~i / /' \'' It al'/ li *1M -,_ - JOY gm Ai i n. on Fig. 7. Number of megabytes transmitted. While these curves, with one notable exception, show pro- available at all three of the original nodes. The Merit-Telenet gressively increasing usage, it must be recognized that the internetwork tie started in December 1976 using links to network has matured appreciably during this ten-year period. Hermes ports. In early 1981 the X.25 service was introduced. New services have been introduced and existing ones improved. The WMU node began to provide Hermes service in August Certainly the current network is a much more powerful and 1979. Their host became available for interactive access in convenient one than the original 1972 network. It also involves January 1981. The impact of this development sequence is more hosts and three more communications computers. reflected in data shown in the five plots. The previous point can bbe tter understood by briefly The growth exception previously cited is the heavy data tracing the network's development history; for more details traffic in Summer 1981 as seen in Figs. 6 and 7. It was due to see [2]. Merit first offered only host-to-host interactive service, a large flow of print jobs from the UM to WSU to a Xerox IT connections, between two of notaes in July 1972. This 900 laser printer. This exceptional traffic ceased when the service was extended to th include aternetwork thirie started in DOctoember 1972. U Computusing nter acquired its own Xeox 9to700. In Octorker 1973 the original 2 during its/s inten-year periode telephone A second way to represent the X.25 setwork's growth is to show lines were replaced with 800 bits/s service. atch service fimproved.t the same data used in the previous plots i a hiugust. appeared in October 1974 between two nodes. at was not fully ig. 8 shows the monthly average number of inusers for each of extended to the third node until January 1977. Hermes service the ten years measure from Inbound connections. Figs. 9-12 began in a limited way at one node in September of 197. Thisa show r each of he other network usage meaintroduced the TL connection type. By May 1977, Hermes was sures.

AU1PPER.: MeTS EVOUTTION 889 300 2000.0............ 4.oo.. 60, ~~~~~~a ~=~~120';,soo' OQ E~1 l~ll 6~MI ao'o 0 0 0 172 1t97 1076 1676 100o 172 1974 1097 1970 1980 1973 1975 1977 1979 19t1 1973 175 1977 1979 1981 Toorso iYor Fig. 8. Yearly averages of number of different network users per month. Fig. 10. Elapsed connect time per year (hours). V. NORMALIZED AND RATE-RELATED NETWORK Similarly, the values for transmitted packets are given in units STATISTICS of single packets rather than kilopackets and those for transmitted characters in kilobytes instead of megabytes. Yearly An alternative way to treat the data presented in the last averages for the same data are given as histograms in Figs. section is to divide the three traffic measures-connect time, 16-18. packets, and bytes-by the corresponding number of monthly It is interesting to note the initially nearly constant but more connections. This normalization yields average values of these recently increasing value for packets per connection, the steady key network performance measures on a per-connection and somewhat more rapidly increasing value for kilobytes per basis. connection, and the apparently bimodal characteristics of time The next set of plots, Figs. 13-15, shows Merit's normalized per connection. These changes are due to a pair of causes that Inbound time-series data for the ten-year period July 1972 to affect these aggregate statistics in opposite ways. As is subJune 1982. Dashed seventh-degree polynomial curves again sequently demonstrated in Section VI, the batch-type conare drawn through the data. In contrast with Section IV, nections-EX and PR-are characterized by very short avconnect time is here expressed in minutes rather than hours. erage connect times and very large values for packets and bytes S00 250 1100 12 17 200 Yor. Tor-. 9. Number of ction per year.. 1 Number of ilopacets transmitted per year. * 0 o 0o C300'150 0200 1too ~0t I so 1672 16711 1676 1676 1660 1672 16716 1067 1067 1080 1673 1675 1677 1670 1961 1673 1975 1677 1679 1961 Fig. 9. Number of' succaful onnections per year. Fig. Ii. Number of kilopackets transmitted per year.

890 IEEE TRANSACTIONS ON COMPUTERS, VOL C-32, NO. 10, OCTOBER 1983 1'2 - _.......... good explanation has been proposed for this consistency over Merit's relatively varied operational circumstances. This fact should interest network designers, some of whom have been known to spend hours debating what a network's maximum e 9 i WI packet length should be. o I 310 VI. NODE- AND TYPE-SPECIFIC DATA v | [ M1 a [ In the two previous sections data were presented in an age agregate form, combining all nodes and connection types. In this - 1 KX B section, the emphasis shifts to the distribution of network aco at Ij 0 8W8 M | tivity among the nodes and to the differences in the various Sz~~~~~~ s m,~ 2^~ >~ X measures when analyzed by their respective connection types. X [ >0 a >o<91 Only data from our last fiscal year, July 1981 through June 1982, are used here: these data most accurately portray the network's recent utilization and best represent its diversity of 0 L_ ~ g lo j |9l ^Qs I<services. Only averages based on these twelve months of In1972 t1974 1976 1978 1980 bound data appear; the monthly fluctuations are not shown. 1973 t975 1977 1979 191t The distribution of usage varies considerably among the Tr~or hosts. This may be seen by examining the bar graph in Fig. 25. Fig. 12. Number of megabytes transmitted per year. It shows the relative number of connections, elapsed hours of connection time, kilopackets, and megabytes per host. The relative newness and still limited network access of the Western transmitted per connection. In contrast, the Hermes (TL) and Michigan host is reflected by their comparatively low usage. Telenet (X3) type of connections exhibits just the inverse set In contrast, over half of the network's traffic flows to Wayne of characteristics. There are now significantly more of these State University's two hosts. direct, terminal-initiated connections than any other type. This Like the distribution among the hosts, the distribution of explains the increase in the average connect time starting in usage among the five connection types is also very unbalanced. the latter half of 1975 and apparently ending in 1977 as the Fig. 26 shows that terminal connections dominate all others. network's mix of connect type usage began to stabilize. Al- This bar chart clearly demonstrates that conventional terminal though the number of batch connections is fewer, the great access to hosts is by far the most-used Merit network service; amount of data that they carry explains the rise in the other it even accounts for about 40 percent of the total data transtwo measures. The observed effect of the relatively heavy print mitted. traffic in the Summer 1981 is consistent with this explana- The next topic addressed is how connect time, packets, and tion. bytes per connection vary with connection type. One expects As with the normalized statistics, it is possible to compute connect time to be greater for interactive than for batch conother ratios from the basic Inbound data. For example, the two nections. But how much greater? How will packets and bytes rate parameters-bytes per second and packets per hour-and vary among the several connection types? These questions are the number of bytes per packet are useful to network designers. answered in the next series of figures. The time-series plots and linear regression curves for these Fig. 27 shows the variation of connect time per connection three parameters appear in Figs. 19-21. Yearly averages for as a function of the five connection types. In this figure a solid them are shown as histograms in Figs. 22-24. line joins the average values but has no other significance. As The large spike which appears in Figs. 19 and 20 for October expected, the host-to-host interactive, terminal, and external 1973 is attributable to extensive testing of the network's connections have much greater average connect times than do transmission capabilities by its staff. While this data point is the batch services. In fact, the average connect time of a print atypical, it also demonstrates that much higher throughput connection is only 2.6 minutes compared with more than 29 rates are possible than those which occur naturally in the minutes for a terminal connection. The difference between network. The generally declining average values for bytes per terminal and external connect time is interesting to note. It second and packets per hour until mid- 1977 reflect the influ- may be due to the higher cost of using external access. ence of Hermes connections, with their lower rate of data The shaded box surrounding each average conveys two transmission. The rise in these parameters since then is due to additional pieces of information. The vertical extremes corthree factors. One is the increasing use of the network's batch respond to plus and minus one standard deviation from the service to transfer large data files, a second is the fact that more average value. The width of each box is proportional to Zite interactive text-processing and computer conferencing are number of connections for its connection type. being done, and a third is that more 1200 bits/s terminals now The next five figures (Figs. 28-32) also use the graphical use the network. format described just above. They show respectively the A remarkable fact is that the number of bytes per packet packets and kilobytes per connection and the number of bytes has remained almost constant over this ten-year period. No per packet, bytes per second, and kilopackets per hour.

AUPPERLE: MERITS EVOLUTION 891 ~~~~~s —- t ^ I IO 91 el 8 t (S31nNIW) NOIi33NNO3/3WIl 133NN03 3i I~~~~~~~~~~~~~~~~~~~~~~~~~L U~~~~~~~~ af )t Ot 91 tl 8 O~~ [S~~lnNIW] NOI133NN03/3WII 133NN03~~~

892 tEEE TRANSACTIONS ON COMPUTERS. VOL. C-32, NO. 10, OCTOBER 1983 r*~ ~ ~~\ *s.010~~~~~~~~~~~~~~~~~~~~~~~~ I s Cai ^-*^'~~~~~~~~r I' Eo r 3 009 OOS 00^ 00~ 002 0011 0 NOI13NN3NN03/S13t3U

893 AUPnPLLE: MERT'S EVOLUTION'-'0 B 0t SN0/SlSOI =-^-^T ~ ~ ~ < ob Fl at aIOo S~ i SU ~ NO~~~i33NNO3/S31)%8OlI>I~ ~~~~c

894 IEEE TRANSACTIONS ON COMPUTERS, VOL. C-32. NO. 10, OCTOBER 1983 9.. - - I 0. 0 0 * g 12972 97 979 connections, as revealed in Figs. 28 and 29, is readily ex-in earlier years. 500._________________________________.. INDIVDUAL PARAETER VARIATIONS mi if This section lists the distribution of seveal ne work mea400. ayo~ m m|suresasedontheindividual ohly statistcal eors rather Se m tablescontain nunddata oron mon. frs or row liststheparametrvaluesf all the 1972% 197 1978 10 1972 1971 1te7 a97s 1eso 1973 1975 1977 1979 1981 1 73 1975 1 77 th979 1st Yearo eoeare Fig. 16. Yearly average of connect time per connection (minutes). Fig. 18. Yearly average of kilobytes per connection. That significantly more data is carried by individual batch ratios for Execute connections compared with those observct connections, as revealed in Figs. 28 and 29, is readily ex- in earlier years. plainable. Since the network makes it convenient to route The batch transmission rates shown in Figs. 31 and 32 are output to a remote printer, most users prefer to use this service limited primarily by the data rates on the network links. While for large print files rather than copying such output to their interactive and batch connections have equal priority within terminals. Similarly, for reasons of both economy and conve- the network, the hosts limit the number of concurrent connience users use the Execute connection type to copy large files nections of each class. Typically each host will accept only one between hosts.,This latter service has been made even more batch connection from each other host but will accept many cost-effective and easy with the introduction of a special batch concurrent interactive connections. This form of regulation file-copy utility program which packs data into maximum has allowed batch traffic to move at maximum data rates (240-byte) packets. This utility program accounts for the in- without resulting in any network-caused response delays for crease in the kilobytes per connection and bytes per packet interactive traffic. The significant difference between the interactive and batch average data rates makes this mix possible. 500 VII. INDIVIDUAL PARAMETER VARIATIONS This section lists the distribution of several network mea1oo0 sures based on the individual monthly statistical records rather than on the monthly aggregates. These data best descr;be the variatiori among the network's connection types. The following 300 tables contain Inbound data for one month. The fiust or All row ~~~c~~~~~~~~ ~~~~lists the parameter values for all of the individual connection C o0~~~~~~~~~ ~~~~types treated as a group. Since some minor connection types 200,,~~~~~~~~~~~ ~are not described in this paper, the All totals slightly exceed f~~~~~~~~~~~~200 ~the sum of the individually-listed connection types. ~aQ I~W~8~~ ~~ CII I~ ~ 01VIII. CONCLUDING REMARKS 10O0 What are the purposes and reasons for networking? There are many, but the four most commonly given are: remote ac1casS, load sharing, resource sharing, and process sharing. Each 1972 -197 9 7 190o of these offers either economic or other advantages. t$?s le7s re1? 97te 1979 1The first of these, remote access, means utilizing distant oeere computing services through some form'f communication Fig. 17. Yearly average of packets per connection. system. Most commercial cor, puter timesharing companies

AUPPiRLE: MERITS EVOLUTION 895 *" o ~L~~~~~'4 U --— 4 ON0O3S/S31lA8 I -^ S /> ~o &~ y " S <^^ I - ^^ s^ |~~~~~C <^^ / 11~~~ ^^ — ""'""^^ <s~~h -*____ \L ~ —---- i —--— i______,______, < ^ 1______~ ~ ~ 8fr 0^ 26?2Sl^ ------ ^0~~~~~~~~~~~~ ON033S/S31Ae~~~~~

8IEEE TRANSACTIONS ON COMPUTERS, VOL. C-32, NO. 10, OCTOBER 1983 /1' i II -I I t oooe oost ooot oos O oooi oos o dnOH/Si3XOUd

AUrr1l:^: MIrch EVOLUTION 897 Y is ~~~t/~~~~~~~~~t t tr ^^^-^^ <" ro X 133yd/S3lA8

IEEE TRANSACTIONS ON COMPUTERS, VOL. C-32 NO. 10, OCTOBER 1983 20 soa__ __ __50 197 74 1976 1978 1980972 1974 97 197 980 c12a~~~~g~~ ~ 0Q~30 o 0 1972 19714 1976 1978 1980 1972 19711 1976 1978 1980 1973 1975 1977 1979 1981 1973 1975 1977 1979 1981 Tear TYears Fig. 22. Yearly average number of bytes per second. Fig. 24. Yearly average number of bytes per packet. LAN networks. Load sharing has never been successfully offer remote access to their clients through such a network. employd heterogeneous networks This allows them to operate a single central facility and yet eore hring s mang aaae t a serve clients throughout the country or even world. These serve clients throughout the country or even world. These base of users all or many of the network's collective resources. companies find remote access to be a cost-effective alternative Resource-sharing networks typically allow users access to when compared with maintaining a distributed set of com- unique hardware, data bases, programs, and pro puting facilities. a g l a t gramming languages that are not provided by the user's local Load sharing implies distributing a given load among the resource-sharing network, several computers on a network. This seemingly obvious and and s. ere areoreharn netor desirable objective is surprisingly difficult to implement. While Process sharing combines and extends the features of loadeffective load sharing is more easily accomplished when all of resource-sharing. In a processsharing network all of its and resource-sharing. In a process-sharing network all of its a network's computers use common hardware and software, automatically managed on a global scale so that it was rarely attempted until the last year or two with some aa s and proces (programs) can migrate throughout data files and processes (programs) c an migrate throughout the network in response to user requirements and the current puting~~~ ~~~~1500 ______________system loading. Individual users may not necessarily care about or be aware of the physical location of their resources in this type of network. Some practical examples of local processsharing networks are now being introduced. 1200 1 1 7 7 19007 11 cn0 I 0e g Fig. 23. Yearly average number of packets per hour.Fig. 25. Relativ traffic to each host. -60 Weetesrn N ohuen — stem #odig Idviu seyneri e State-us wevfe St ele-MU 1972 197q1 1976 1976 1980 ~T. ~. e#oeb~~ee Fig. 23. Y early average number of packets per hour. Fig. 25. Relative traffic to each host.

AUPtWEKLE: MERITK EVOLUTION 899 300 Hlat-&t*-He SOo IS0 TereToeL ~ Fig. 29. Kilobytes by connection per onnection type. so 5ooc tBO 90 t. Euiar.rn.LalH ExISutOo e. r t' - — oFig. 26. Relative conne ction typedistribution. Cenneoiton Tpo, Fig. 29. Kilobytes by connection per connection type. ISO Hood invlved: printcr. B - ^ -^o~~~~~~ I ~s t100~elal IIoo0 llo C'Co' IoPt Ie~.h 50 Tb.. Uieb fxi er e Wst -Lt-oHoeL ~i nn ~raiL PrLnt C.nno.oofn Typei TOrLnoAL CEx~eut Fig. 2. Connect ime per connection by connection type. Cenneoiffn Tgpo. Fig. 30. Bytes per packet by connection type. s 000 50~1 ~ ~ ~ ~ ~ ~ ~ ~ ~ ~~~100 25~~~~~~~~~00~ ~1200 ell0~~~~~~~~~~~~ ~0.8 So Fig. 28. Packets per connection by connection type. Co2n,.Lon Typos

900 IEEE TRANSACTIONS ON COMPUTERS. VOL. C-32, NO. 10, OCTOBER 1983 eo.TABLE I CONNECT TIME PER CONNECTION (HOURS) Connection Number of Mean Standard Maximum 1o Type Connections Value Deviation Value | All 47 218 0.379 0.733 16.624 = j;IT 3607 0.163 0.372 12.910 ~5@~~~ / ~TL 30 291 0.463 0.819 16.624 j0 X3 9642 0.331 0.608 12.900 /SE EX 1597 0.017 0.103 2.450 t I IP/ PR 1766 0.032 0.265 10.847 20 TABLE II PACKETS TRANSMITTED PER CONNECTION.........._. _ 0 L —-. Connection Number of Mean Standard Maximum Mo-teio-io- t Externol Print Type Connections Value Deviation Value Terennol Exrout -- Conn.ets.on TUpe All 47 218 394 1163 55845 Fig. 32. Kilopackets per hour by connection type. IT 3607 293 1364 55 845 TL 30 291 386 914 34 997 X3 9642 315 815 22 581 EX 1597 453 2108 48112 2) File Sharing: The ability to read from, write to, or PR 1766 1173 3111 49535 update files on a remotely located computing system as if they were local.TABLE TABLE Iii 3) Program Sharing: The ability either to send a loadable KILOBYTES TRANSMITTED PER CONNECTION program to a remotely located computing system to be loaded and executed by that system, or to execute a remotely located Connection Number of Man Standard Maxium Type Connections Value Deviation Value program with local data. 4) Interprocess Sharing: The provision of data paths be- T 4 7 164 613 tween programs running on separate hosts, so that large tasks TL 30 291 14 35 2091 may be divided into smaller units for execution at different X3 9642 13 37 1115 computing sites in the network. EX 159 39 262 6132 * ~.. EX 1597 39 262 6132 computing sites In the network. PR 1766 85 270 4211 Merit is primarily a resource-sharing network; it is not, nor for a long time is it likely to become, either a load- or process-sharing network. In varying degrees Merit has imple- periences of others since little similar information has been mented all four of the sharing modes described above. But the published. Perhaps this paper will stimulate others to provide onus is still on users to know where resources are located, how corresponding usage data about their networks. they are used, and how these resources are accessed through Throughout this paper comments amplifying the data have the network. A user cannot enter the network and simply re- been made where it seemed appropriate to do so. A few addiquest "run my Pascal program" with the expectation that the tional, more general, ones, have been left for this section. The network will handle all of the necessary details. Instead, users first of these addresses how individuals make use of the netmust specify which node they wish to access, sign on or log in work. to that host, and issue the necessary commands, in the selected Without question the most helpful network services are the host's own command language, to achieve their computing Hermes (direct terminal-to-host) and external (X.25 link with task. This differs from IBM's SNA approach to networking, Telenet) services. The introduction of these services, more than for example. any other single factor, has boosted the network's lev,-l of While users have the responsibility for managing their own usage. The obvious inference is that most of the networking resource-sharing, they are by the same mechanism afforded needs of our users are very simple. They want to access a rethe opportunity to load-share. They may, from time to time, motely-located resource interactively. The network provides direct some of their work to less busy hosts or they may regu- them with an alternative to placing a long-distance telephone larly use a remote host's resources for load- rather than re- call, which, perhaps, they would not make. A further indication source-sharing reasons. Such user-scheduled load sharing does of the importance of these services is given in Table IV. This not account for much network traffic. table lists the hours of Hermes and Telenet use per month The purpose in further elaborating on the typical Merit during the July 1, 1981 to June 30, 1982 fiscal year. The times user's viewpoint is to provide a better basis for assessing the reported are total usage hours, including access through statistics contained herein. Certainly the technology, functional Hermes to a node's local host. capabilities, and user environment of existing networks vary While the host-to-host interactive connections are far less greatly. These factors must influence the statistical measures frequently used, they provide an important service to both observed. But it is nearly impossible to comment intelligently novice and advanced network users. Novices will usually use on how typical or atypical Merit's data are relative to the ex- this service to transfer a data file between hosts. For some, a

AUPPERLE MERITS EVALUATION 901 TABLE IV but individual lines of information from terminals. Likewise, FISCAL 1981 HERMES USAGE IN HOURSI FISCAL 1981 HERMES USAGE IN HOURS3 output is typically a series of printer lines or terminal output From From From From From lines. Therefore, most input lines contain fewer than 80 Month Ann Arbor Detroit E. Lansing Kalamazoo Telenet Total characters and most output lines fewer than 135 characters. July 2069 5503 1267 396 2726 11 961 For this reason, it is not at all surprising that the packets Aug. 1811 4961 1144 334 2579 10829 transmitted over the network average between 35 and 115 Sept. 1999 5198 1089 294 2562 11142. Oct. 2544 6844 1304 226 2422 13 340 e Nov. 2202 6808 1244 175 2507 12936 It should be noted that while Merit's maximum packet size Dec. 2328 6185 1240 192 2278 12223 is 240 bytes, it is possible to send very long records as strings Jan. 2392 66056 1183 384 2548 12 563 of maximum-length packets. For example, a single record of Feb. 2863 6946 1139 592 3045 14585 Mar. 3602 8164 1173 827 3191 16 957 any length up to 32 767 bytes may be transmitted between the Apr. 3307 8484 1097 871 3192 16951 MTS hosts. Evidently few long records are sent, but this caMay 2703 7067 1154 787 2870 14 581 June 2849 719065 6 8 7 2813 14693 pability is exercised whenever object files are transferred be-........ tween hosts and for some other purposes as well. This ability to ship long records partially account for the somewhat higher single transfer may even be their only use of the network. At byte per packet ratio for IT connections as contrasted with TL the other extreme are individuals who make very clever uses connections. of the IT connection type to perform such tasks as distributed T o o o c o o o.* ~~. w~~~. "'~~~The ratio of either bytes or packets to the duration of a multiprocessing. iroces.. -. connection demonstrates the dramatic superiority in transThe batch services were not initially available and saw only mission rate achieved by batch connections, both EX and PR limited use when they were introduced, but the number of these'. *.. ^..types, over the much slower interactive connections. The relconnections continues to increase. To some users they are very ative throughput rate differs by more than an order of magimportant. For example, the resources on Wayne State Uninitude, as shown in Figs. 31 and 32. The primary reason for versity's MVS system are accessible only in batch mode. An-..i.. i a r mn this difference is surely that I/O from an interactive terminal other reason for interest in batch is that it is easier to use a s c s t ic is much slower than interhost, machine-controlled data foreign system in this fashion than it is to become an effective transfers. But there are also some technical factors related to interactive user. This-is particularly true for a classroom apinter e ur. Ttic y te fr a c m ap- Merit's host-to-host protocols and the use of data compression plication, where a course instructor can easily set up a specific w n te n k t i t d i r. within the network that influence this difference in rates. set of commands to compile and execute a source program for -..,,.,...,,~ -Merit's average packet per hour rate for TL connections is his class on a remote host. In cases like this, the students need. 890 and for X3 is 998. These rates compare closely with the very little, if any, understanding of another computer's com-. r'and language, editing system, hand file system\kilopacket per hour rate experienced by some other packetmand language, editing system, and file system.switching networks A final note about batch services is that, in addition to the A factor of frequent concern to users is how responsive a execute and print services described and reported on earlier, e. s computing system is. There are many ways to quantify rethere is a third service that carries data to be punched. Al- sponse performance. For example, the National Bureau of though this service has been available just as long as the other Standards [31 has defined various performance measures and batch services, only a handful of punch connections are opened conducted some experiments related to this issue. Merit's exeach month. Evidently our users have little use for cards; this is n e. perience has been that for typical interactive computing sesis not a great surprise. sions the network does not introduce noticeable additional Another set of observations to make about Merit's statistical response delays. Any user-observable response sluggishness experiences pertains to the relationships between packets,, 6 erie s prtais to te retio is bt n p t is attributable to a host system. On the other hand, a network bytes, and the length of network connections. The average file transfer between hosts takes much longer than would a number of bytes per packet is about 50 when all connection comparable intrahost data transfer. This delay is due in part classes are aggregated. When the various connection types are to the relatively low-speed data lines linking the Communiconsidered individually, the averages vary from a low of about cations Computers, and in part to idiosyncrasies of the sup35 for the interactive TL and X3 types to over 115 for the batch porting interactive protocols in the hosts, and for this reason EX type, as shown in Fig. 30. But all of these averages are mu ch, sall thawn Meit'.s maximum pake size ofrae 2A4 users are advised to transmit long files between hosts using the much smaller than Merit's maximum packet size of 240 network's batch services. In any case, Merit has not attempted Whst. cue ths partcula.. Agi teasriseither to systematically record or to seriously study response What causes these particular values? Again the answer is performance. fairly obvious. All of the hosts on the network except for WMU's DEC system and the newly added VAX use recordoriented input and output rather than character-stream 1/O. REFERENCES This means that input records are card images or successive [1] W. D. Becher and E. M. Aupperle, "The communications hardware of the Merit computer network," IEEE Trans. Commun., vol. COM-20, 3 As of June 1981 there were 32 ports in Detroit; 17 ports in Ann Arbor; June 1972. 10 ports in East Lansing; and 8 pos in Kalamazoo. Telenet is serviced through 12] E. M. Aupperle, "Merit computer network experiences," in Proc. Int. an X.25 synchronous link capable of supporting 32 concurrent inbound con- Cof. Commun. Cof. Record, vol. 111, June 16-18, 1975, pp. 33-2nections. 33-6.

902 IEEE TRANSACTIONS ON COMPUTERS, VOL C-32, NO. 10, OCTOBER 1983 [31 M. D. Abrams, S. Treu, and R. P. Blanc, "Measurement of computer search Enineer, from 1963 to 1967 an Asociate Research Engineer, and from communications networks," NBS Tech. Note 908, July, 1976. 1967 to 1969 a Research Engineer at Cooley Electronics Laboratory, the [4] E. M. Aupperle, "Mini and microcomputers and computer communi- University of Michigan, working with fuze countermeasures, application of cation," The Interational Computer Technology Conference, ASME solid-state devices, and pseudorandom communication system counterCentury I —Emerging Technology Conferences, Advances in Computer measures. He has been a Lecturer with the Department of Electrical and Technology- 1980, vol. I, pp. 265-270, Aug. 12-15, 1980. Computer Engineering since 1963 and a Research Scientist with the Institute of Science and Technology since 1969 at the University of Michigan. His present responsibilities include serving as Director for the Merit Computer Network, a joint interuniversity computer network activity of Michigan State ap:8~; Eric M. A erk (S'57-M'61 -SM'65) was born University, East Lansing, the University of Michigan, Ann Arbor, Wayne -~"?'? _ in Batavia, NY, on April 14, 1935. He received the tatUniversity, Detroit, and Western Michigan University, Kalamazoo. B.S.E. degree in electrical engineering, and the He is also an Associate Director for Communications at the University of B.S.E. degree in mathematics in 1957, the M.S.E. Michigan Computing Center. degree in nuclear engineering in 1958, and the Mr. Aupperle is a member of Phi Eta Sigma, Eta Kappa Nu, Tau Beta Phi, lnstm. E. (the professional degree in engineering Sigma Xi, Phi Kappa Phi, and the University of Michigan Science Research - granted by the Program in Computer, Informa- Club. He also has held several offices in the National Electronics Conference. tion, and Control Engineering, formerly the pro- In addition, he has acted as Chairman and faculty member of the University gram of Instrumentation Engineering) in 1964, all of Michigan Engineering Summer Conference's short course on minicomfrom the University of Michigan, Ann Arbor. puters for several years. He is a Registered Professional Engineer in the State From 1957 to 1963 he was an Assistant Re- of Michigan. UNIVERSITY OF MICHIGAN II3 9111111 115 02493 8337IIII l 11 I1111 3 9015 02493 8337