Performance effects of systems analysis methods: An experimental evaluation.

Abstract

The acquisition of accurate and complete business knowledge is one major success factor in developing an information system. This research focuses on whether and why systems analysis methods facilitate cognitive information processing for the acquisition of business knowledge. This research evaluated two popular structured analysis methods, Data Flow Diagramming (DFD) method and Jackson System Diagramming (JSD) method, on a theoretical basis and tested empirically whether they provide beneficial effects. Both methods organize information in ways that studies in text comprehension have found to be helpful for better knowledge acquisition. The DFD method has the analyst organize information hierarchically starting from general information to more specific information; in contrast, the JSD method encourages the analyst to organize information in a temporal order focusing on more specific information. Hence, analysts using either method would acquire better business knowledge. In a laboratory experiment, 55 BBA students were randomly assigned to three groups: DFD, JSD, and Free. The Free group, a control group, was asked to draw diagrams of their own devising. All subjects were given the same business case, asked to construct diagrams describing the business, and then given a memory test after two or three days. The memory test measured the knowledge attained by a subject. The study showed that both method groups acquired more accurate and complete knowledge than the Free group. In particular, the DFD group recalled a significantly higher number of facts than the Free group. Two advantages of the DFD method are: the DFD method forces one to see an overview of the business and its environment; and it recommends that the analyst describe high-level activities before low-level activities. In general, high-level information is more recallable and plays the role of retrieval cues which remind associated low-level information. However, there was no significant difference in business knowledge between DFD and JSD. The Free group liked their own way of diagramming more than the method groups, but recalled many fewer items later, even though the Free group made diagrams containing much more information than the method groups. These findings emphasize the importance of having a systems analysis method which organizes information in a structured way. The mere recording of a larger quantity of information does not benefit later understanding. This implies that businesses and academic institutes need to teach a structured analysis method to prospective systems analysts. This study contributes a better understanding of how systems analysis methods provide benefits in acquiring business knowledge. We also suggest a few ways for improving the current systems analysis methods.