EDGS 646 Index

The Newman Hierarchy of Error Causes for Written Mathematical Tasks

Since 1977, when the Australian educator M. Anne Newman published data based on a system she had developed for analysing errors made on written tasks (see Newman, 1977a, b), a steady stream of research papers has been published in many countries in which data from many countries have been reported and analysed along lines suggested by Newman (see, for example, Casey, 1978; Clarkson, 1980, 1982, 1991; Clements, 1980, 1982; Marinas & Clements, 1990; Watson, 1980).

The findings of these studies have been sufficiently different from those produced by other error analysis procedures (for example, Hollander, 1978; Lankford, 1974; Radatz, 1979), to attract considerable attention from both the international body of mathematics education researchers (see, for example, Dickson, Brown & Gibson, 1984; Mellin-Olsen, 1987; Zepp, 1989) and teachers of mathematics. In particular, analyses of data based on the Newman procedure have drawn special attention to (a) the influence of language factors on mathematics learning; and (b) the inappropriateness of many "remedial" mathematics programs in schools in which there is an over-emphasis on the revision of standard algorithms (Clarke, 1989).

The Newman Procedure

According to Newman (1977, 1983), a person wishing to obtain a correct solution to a one-step word problem such as "The marked price of a book was $20. However, at a sale, 20% discount was given. How much discount was this?", must ultimately proceed according to the following hierarchy:

1. Read the problem;

2. Comprehend what is read;

3. Carry out a mental transformation from the words of the question to the selection of an appropriate mathematical strategy;

4. Apply the process skills demanded by the selected strategy; and

5. Encode the answer in an acceptable written form.

Newman used the word "hierarchy" because she reasoned that failure at any level of the above sequence prevents problem solvers from obtaining satisfactory solutions (unless by chance they arrive at correct solutions by faulty reasoning).

Of course, as Casey (1978) pointed out, problem solvers often return to lower stages of the hierarchy when attempting to solve problems. (For example, in the middle of a complicated calculation someone might decide to reread the question to check whether all relevant information has been taken into account.) However, even if some of the steps are revisited during the problem-solving process, the Newman hierarchy provides a fundamental framework for the sequencing of essential steps.

Clements (1980) illustrated the Newman technique with the diagram shown in Figure 1. According to Clements (1980, p. 4), errors due to the form of the question are essentially different from those in the other categories shown in Figure 1 because the source of difficulty resides fundamentally in the question itself rather than in the interaction between the problem solver and the question. This distinction is represented in Figure 1 by the category labelled "Question Form" being placed beside the five-stage hierarchy. Two other categories, "Carelessness" and "Motivation," have also been shown as separate from the hierarchy although, as indicated, these types of errors can occur at any stage of the problem-solving process. A careless error, for example, could be a reading error, a comprehension error, and so on. Similarly, someone who had read, comprehended and worked out an appropriate strategy fir solving a problem might decline to proceed further in the hierarchy because of a lack of motivation. (For example, a problem-solver might exclaim: "What a trivial problem. It's not worth going any further.")

Figure 1. The Newman hierarchy of error causes (from Clements, 1980, p. 4).

Newman (1983b, p. 11) recommended that the following "questions" or requests be used in interviews that are carried out in order to classify students' errors on written mathematical tasks:

1. Please read the question to me. (Reading)

2. Tell me what the question is asking you to do. (Comprehension)

3. Tell me a method you can use to find and answer to the question. (Transformation)

4. Show me how you worked out the answer to the question. Explain to me what you are doing as you do it. (Process Skills)

5. Now write down your answer to the question. (Encoding)

If pupils who originally got a question wrong get it right when asked by an interviewer them to do it once again, the interviewer should still make the five requests in order to obtain information on whether the original error could be attributed to carelessness or motivational factors.

Mellin-Olsen (1987, p. 150) suggested that although the Newman hierarchy was helpful for the teacher, it could conflict with an educator's aspiration "that the learner ought to experience her own capability by developing her own methods and ways." We would maintain that there is no conflict as the Newman hierarchy is not a learning hierarchy in the strict Gagné (1967) sense of that expression. Newman's framework for the analysis of errors was not put forward as a rigid information processing model of problem solving. The framework was meant to complement rather than to challenge descriptions of problem-solving processes such as those offered by Polya (1973). With the Newman approach the researcher is attempting to stand back and observe an individual's problem-solving efforts from a coordinated perspective; Polya (1973) on the other hand, was most interested in elaborating the richness of what Newman termed Comprehension and Transformation.

The versatility of the Newman procedure can be seen in the following interview reported by Ferrer (1991). The student interviewed was an 11-year-old Malaysian primary school girl who had given the response "All" to the question "My brother and I ate a pizza today. I ate only one quarter of the pizza, but my brother ate two-thirds. How much of the pizza did we eat?" After the student had read the question correctly to the interviewer, the following dialogue took place. (In the transcript, "I" stands for Interviewer, and "S" for Student.)

I: What is the question asking you to do?

S: Uhmm . . . It's asking you how many . . . how much of the pizza we ate in total?

I: Alright. How did you work that out?

S: By drawing a pizza out ... and by drawing a quarter of it and then make a two-thirds.

I: What sort of sum is it?

S. A problem sum!

I: Is it adding or subtracting or multiplying or dividing?

S: Adding.

I: Could you show me how you worked it out? You said you did a diagram. Could you show me how you did it and what the diagram was?

S: (Draws the diagram in Figure 1A.) I ate one-quarter of the pizza (draws a quarter*).

Figure 1. Diagrammatic representations of the pizza problem.

I: Which is the quarter?

S: This one. (Points to the appropriate region and labels it 1/4.)

I: How do you know that's a quarter?

S: Because it's one-fourth of the pizza. Then I drew up two-thirds, which my brother ate. (Draws line x - see Figure 1B - and labels each part 1/3)

I: And that's 1/3 and that's 1/3. How do you know it's 1/3.

S: Because it's a third of a pizza.

(From Ferrer, 1991, p. 2)

The interview continued beyond this point, but it was clear from what had been said that the original error should be classified as a Transformation error--the student comprehended the question, but did not succeed in developing an appropriate strategy. Although the interview was conducted according to the Newman procedure, the interviewer was able to identify some of the student's difficulties without forcing her along a solution path she had not chosen.

Summary of Findings of Early Newman Studies

In her initial study, Newman (1977a) found that Reading, Comprehension, and Transformation errors made by 124 low-achieving Grade 6 pupils accounted for 13%, 22% and 12% respectively of all errors made. Thus, almost half the errors made occurred before the application of process skills. Studies carried out with primary and junior secondary school children by Clements (1980), Watson (1980), and Clarkson (1983) obtained similar results, with about 50% of errors first occurring at the Reading, Comprehension or Transformation stages. Clements's sample included 726 children in Grades 5 to 7 in Melbourne, Watson's study was confined to a preparatory grade in primary school, and Clarkson's sample consisted of 95 Grade 6 students in two community schools in Papua New Guinea.

The consistency of the results emphasised the robustness of the Newman approach, and drew attention to the importance of language factors in mathematics learning. If about 50% of errors made on written mathematical tasks occurred before the application of process skills, then, clearly, remedial mathematics programs needed to pay particular attention to whether the children were able to comprehend the mathematics word problems they were being asked to solve.

Some Recent Newman Data

The original studies by Newman (1977), Casey (1978), Clements (1980), Clarkson (1980) and Watson (1980) were carried out Australia in the late 1970s, but since the early 1980s the Newman approach to error analysis has increasingly been used outside Australia. Clements (1982) and Clarkson (1983) applied Newman techniques in error analysis research carried out in Papua New Guinea in the early 1980s, and more recently the methods have been applied to mathematics and science education research studies in Brunei (Mohidin, 1991), India (Kaushil, Sajjin Singh & Clements, 1985), Indonesia (Ora, 1992), Malaysia (Marinas & Clements, 1990; Kownan, 1992), Papua New Guinea (Clarkson, 1991); the Philippines (Jimenez, 1992), and Thailand (Singhatat, 1991; Sobhachit, 1991). With the exception of the early study by Casey (1988), in each of these studies individual students were interviewed and errors classified according to the first break-down point on the Newman hierarchy. With the Casey study, the interviewer helped students over early break-down points to see if they were then able to proceed further towards satisfactory solutions.

The Newman approach was first used in Malaysia in 1990, when Marinas and Clements (1990) found that over 90% of initial errors made by a sample of Grade 7 students from Penang were of the comprehension or transformation type. A number of other studies based in Southeast Asian contexts have also been carried out, in both mathematics and science education settings.

Clarkson (1991) has examined the relationship between careless/unknown errors, as defined by Newman (1983), and various achievement and psychological measures. In a comparison between the errors made by PNG and Western students, Clarkson concluded that the apparently lower occurrence of careless/unknown errors for PNG students could be attributed to a higher systematic error rate.

Faulkner (1992) has used Newman techniques in research investigating the errors made by nurses undergoing a calculation audit . She found that the majority of errors the nurses made were of the comprehension or transformation type. This result, based on adult data, is interesting in that it extends and confirms the findings of recent research (see, for example, Clarkson, 1991; Marinas & Clements, 1990) that a deeper understanding of the sources of the comprehension and transformation categories of errors is vital.

Ellerton and Clements (1996) carried out Newman interviews with 116 Year 8 students, in 12 classes in 5 schools in New South Wales and Victoria. Despite the fact that all the teachers of the students agreed that their students should not have had difficulty comprehending the written tasks--half of which were in multiple-choice form, and the other half in short-answer form-- Ellerton and Clements found that that 80% of errors first occurred at the Reading, Comprehension and Transformation stages. Only 6% of errors first occurred at the Process Skills stage.

The Ellerton and Clements (1996) study was different from previous Newman studies in that the researchers interviewed students for all questions, including those for which correct responses had been given. In fact, the Newman interviews revealed that for about one-fourth of the correct responses which the students gave they not have a complete grasp of the concepts and skills which the questions were testing. In such cases Newman error categories were attached to these "correct" responses

One last aspect of the Ellerton and Clements (1996) study is of interest. They reported that different questions produced quite different error patterns. Thus, for example, for the following question, 40% of the errors were of the Process Skills variety, and only 15% were in the Reading or Comprehension or Transformation categories:

Ice-creams cost 85 cents each, and apples cost 45 cents each. How much altogether would 7 ice-creams and 5 apples cost?

By contrast, however, for the following question, only 6% of the errors were of the Process Skills variety, but 90% were in the Reading or Comprehension or Transformation categories:

Arrange the following fractions in order of size from smallest to largest:

1/3 , 1/4 , 2/5

The Ellerton and Clements (1996) variations in Newman research methodology --analysing "correct" responses as well as "incorrect" responses, and considering the different error patterns generated by different questions--would appear to have important implications for curriculum and test developers and for classroom teachers. Teachers need to be reminded that many "correct" responses are given by students who do not really understand the concepts being tested. Also, teachers, textbook writers and test developers more aware of the kinds of errors students are likely to make on different kinds of tasks.

The high percentage of Comprehension and Transformation errors found in studies using the Newman procedure in widely differing contexts has, perhaps more than any other body of research, provided unambiguous evidence of the importance of language in the development of mathematical concepts. However, the research raises the difficult issue of what educators can do to improve a learner's comprehension of mathematical text or ability to transform, that is to say, to identify an appropriate sequence of operations that will solve a given word problem. At present, little progress has been made on this issue, and it should be an important focus of the international mathematics education research agenda over the next decade.

References

Casey, D. P. (1978). Failing students: A strategy of error analysis. In P. Costello (Ed.), Aspects of motivation (pp. 295-306). Melbourne: Mathematical Association of Victoria.

Clarke, D. J. (1989). Assessment alternatives in mathematics. Canberra: Curriculum Development Centre.

Clarkson, P. C. (1980). The Newman error analysis - Some extensions. In B. A. Foster (Ed.), Research in mathematics education in Australia 1980 (Vol. 1, pp. 11-22). Hobart: Mathematics Education Research Group of Australia.

Clarkson, P. C. (1983). Types of errors made by Papua New Guinean students. Report No. 26. Lae: Papua New Guinea University of Technology Mathematics Education Centre.

Clarkson, P. C. (1991). Language comprehension errors: A further investigation. Mathematics Education Research Journal, 3 (2), 24-33.

Clements, M. A. (1980). Analysing children's errors on written mathematical tasks. Educational Studies in Mathematics, 11(1), 1-21.

Dickson, L., Brown, M., & Gibson, O. (1984). Children learning mathematics: A teacher's guide to recent research. Oxford: Schools Council.

Ellerton, N. F., & Clements, M. A. (1996, July). Newman error analysis research: Some new directions. Paper presented at the 19th Annual Conference of the Mathematics Education Research Group of Australasia. Melbourne: Mathematics Education Research Group of Australasia.

Faulkner, R. (1992). Research on the number and type of calculation errors made by registered nurses in a major Melbourne teaching hospital. Unpublished M.Ed. research paper.

Hollander, S. K. (1978). A literature review: Thought processes employed in the solution of verbal arithmetic problems. School Science and Mathematics, 78, 327-335.

Jimenez, E. C. (1992). A cross-lingual study of Grade 3 and Grade 5 Filipino children's processing of mathematical word problems. Unpublished manuscript, SEAMEO-RECSAM, Penang.

Kaushil, L. D., Sajjin Singh, & Clements, M. A. (1985). Language factors influencing the learning of mathematics in an English-medium school in Delhi. Delhi: State Institute of Education (Roop Nagar).

Kim, Teoh Sooi (1991). An investigation into three aspects of numeracy among pupils studying in Year three and Year six in two primary schools in Malaysia. Penang: SEAMEO-RECSAM.

Kownan, M. B. (1992). An investigation of Malaysian Form 2 students' misconceptions of force and energy. Unpublished manuscript, SEAMEO-RECSAM, Penang.

Lankford, F. G. (1974). What can a teacher learn about a pupil's thinking through oral interviews? Arithmetic Teacher, 21, 26-32.

Marinas, B., & Clements, M. A. (1990). Understanding the problem: A prerequisite to problem solving in mathematics. Journal for Research in Science and Mathematics Education in Southeast Asia, 13 (1), 14-20.

Mellin-Olsen, S. (1987). The politics of mathematics education. Dordrecht: Reidel.

Mohidin, Hajjah Radiah Haji (1991). An investigation into the difficulties faced by the students of Form 4 SMJA secondary school in transforming short mathematics problems into algebraic form. Penang: SEAMEO-RECSAM.

Newman, M. A. (1977a). An analysis of sixth-grade pupils' errors on written mathematical tasks. In M. A. Clements & J. Foyster (Eds.), Research in mathematics education in Australia, 1977 ( Vol. 2, pp. 269-287). Melbourne: Swinburne College Press.

Newman, M. A. (1977b). An analysis of sixth-grade pupils' errors on written mathematical tasks. Victorian Institute for Educational Research Bulletin, 39, 31-43.

Newman, M. A. (1983). Strategies for diagnosis and remediation. Sydney: Harcourt, Brace Jovanovich.

Ora, M. (1992). An investigation into whether senior secondary physical science students in Indonesia relate their practical work to their theoretical studies. Unpublished manuscript, SEAMEO-RECSAM, Penang.

Radatz, H. (1979). Error analysis in mathematics education. Journal for Research in Mathematics Education, 10, 163-172.

Singhatat, N. (1991). Analysis of mathematics errors of lower secondary pupils in solving word problems. Penang: SEAMEO-RECSAM.

Sobhachit, S. (1991). An investigation into students' understanding of the electrochemical cell and the electrolytic cell. Penang: SEAMEO-RECSAM.

Watson, I. (1980). Investigating errors of beginning mathematicians. Educational Studies in Mathematics, 11(3), 319-329.

Zepp, R. (1989). Language and mathematics education. Hong Kong: API Press.

EDGS 646 Index