EFFECTS OF ENHANCING BEHAVIOR OF STUDENTS AND USE OF FEEDBACK-CORRECTIVE PROCEDURES

Nuray SENEMOĞLU & Ken FOGELMAN

ABSTRACT

     An experiment was carried out to determine the effect of several elements of mastery learning on student achievement in an undergraduate course on curriculum development and instruction, which is a less sequential course than the type of courses used in prior studies. Learning in a less sequential course can be facilitated by previous learning, but the lack of prerequisites does not obstruct learning. Students were randomly assigned to three groups: conventional teaching methods; enhancing cognitive entry behavior plus conventional teaching methods; and feedback/corrective procedures, enhancing cognitive entry behaviors, and conventional teaching methods. The combination of feedback/corrective procedures and initial enhancement of cognitive prerequisites was significantly more effective than using only enhancement of cognitive prerequisites, which in turn was significantly more effective than using conventional methods. The results indicate that using a combination of alterable variables effectively in the teaching-learning process may solve the "two sigma problem" in less sequential subject series and at the university level.

     Bloom's (1976) model of mastery learning explains variation in school learning in terms of three alterable variables. The first is the student's cognitive entry behaviors, that is, the prerequisite learning needed for a particular set of learning tasks. According to Bloom, this variable may explain 50% of the variation in school learning. The second variable is affective entry characteristics that influence the student's motivation to learn the new learning tasks: 25% of variation in school learning may be explained by this variable. The third variable is quality of instruction, which involves the use of cues, participation" of students in instruction, reinforcement, and feedback/correctives. This also may explain 25% of the variation in school learning. When students' entry characteristics and quality of instruction are favorable, ail the learning outcomes should be at a high level and there should be little variation in measures of learning outcomes.

     Research has shown that the average achievement of students who learn under one-to-one tutoring or under one tutor for two or three students simultaneously is two sigmas above the average achievement of students who learn under conventional group methods (typically in a class with 30 students and with tests given periodically only to determine students marks and one sigma above the average of students who learn under mastery learning with feedback/corrective- procedures and are given formative tests '"or feedback and corrective work, followed by parallel formative tests to determine the extent to which die students have mastered ;he subject (Anania. 1981: Burke. 1983). In the last decade. Bloom (1994) and his colleagues have been trying to solve what they call the "two sigma problem." That is they have been seeking more practical conditions than one-to-one tutoring co help students achieve two sigmas higher than students who receive conventional instruction.

     Leyton (1983) suggested chat one approach to the "two sigma problem"' would be to use mastery learning during an advanced course in a sequence in addition to enhancing students initial cognitive entry prerequisites at the beginning of the course. He conducted an experimental test of this method in algebra and French at the high school level. Two classes were helped to relearn the specific prerequisites they lacked. Their achievement was compared with two classes for whom this was not done. The classes that had been retaught the initial prerequisites were approximately 0.7 sigma higher than the other two classes on the first formative test given at the end of a 2-week period of learning tasks in the advanced course. One of these enhanced classes then continued with conventional methods, as the control group, but the other class was also provided with feed-back/corrective procedures over a series of learning tasks. After a 10- v, 12-week period of instruction, this experimental group approximately 1.6 sigmas above the control group or he summative examination.. The average effect or ink enhancement of prerequisites alone was about and the average effect of mastery learning alone was sigma. It appears that the separate effects of initial enhancement and master were additive.

     In a similar study (Sayan 1986) of teaching English as a second language at a private high school, the average effect of initial enhancement of prerequisites alone was about 0.73 sigma and the average effect of mastery learning one was 1.76 sigmas. The average effect of combining the two variables was 2.76 sigmas.

     This solution to [he two sigma problem" is likely to be explicable to sequential courses, that is. where learning each course/task is a prerequisite for success in the next course/task in the series. When alterable variables of mastery learning and entry characteristics are used with such courses and tasks, student achievement can reach mastery level, and variation in achievement can decrease (Block. 1971; Block & Bums; 1976; Bloom, 1976: Clark. Giskey, & Beninga. 1983: Guskey & Monsaas. 1979: Ozcelik. 1974).

     Although much research has been carried out on sequential subjects, there has been no study of the relevance of mastery teaming for less sequential subjects, that is. where the learning of previous content is not required but previous learning facilitates the learning of subsequent content in the series. For example, learning in one course in a less sequential series can facilitate learning in the next course but lack of learning in the first course does not obstruct learning in the later course, as it would in a sequential series. In sequential series, certain behavioral objectives cannot be achieved or learned unless the student has mastered previous learning, In the study described here, we investigated [he effects of enhanced cognitive entry behavior in combination with feedback corrective procedures on overall learning, as well as on learning at the levels of knowledge, comprehension, and application, in a less sequential subject at the university level.

METHOD

     Sample

     This study was conducted with students who were enrolled in a course titled Curriculum Development and Instruction (GDI), offered by the Department of Educational Sciences, Faculty of Education. Hacettepe University. This course is taught in die 2nd year of a 4-year bachelor of education degree. program. The course covers curriculum development, determining objectives and behavioral objectives (described in terms of operational verbs, to increase degree of specificity and observability of objectives), organizing teaching-learning situations, and evaluating student achievement and curriculum. Prerequisites for this course. include three courses: educational psychology, educational sociology, and educational philosophy. Ninety students who took these courses were randomly assigned to three groups. Three lecturers taught the same unit in each group in order [o control for lecturer effects.

     Measures

     All the students were administered the Cognitive Entry Behavior (CEB) Test as a pretest at the beginning of the course. This test was readministered after efforts to enhance cognitive entry behavior. The CEB is composed of three subtests that assess mastery of knowledge required for the Curriculum Development and Instruction course. The knowledge involves content covered in the three prerequisite courses listed above. Two of the learning tasks n the CD1 course concern (a) developing and understanding the concepts of curriculum and instruction and (b) writing objectives and behavioral objectives. Understanding or these concepts facilitated by learning concepts and principles ct educational philosophy. educational school arid educational psychology. In addition, learning tasks related to the organization and management of teaching-learning processes is facilitated by learning the concepts and principles of educational psychology.

     Specialist tutors identified the courses in educational psychology, educational philosophy, and educational sociology as prerequisites for the CD1 course before we constructed the CDI course. At least one item was written to measure each prerequisite. The test was administered to a pilot group and subsequently revised. Reliability coefficients of the subtests using the K-R 20 formula, were 81 for psychology of education. .89 for philosophy of education, and .81 for sociology of education.

     Formative tests were constructed to measure five objectives of the CDI course: (a) recognizing the concepts of curriculum development and instruction, (b) determining objectives and behavioral objectives, (c) organizing teaching-learning situations, (d) teaching methods, and (e) evaluation of the curriculum. For each formative test, objectives and behavioral objectives were determined, and at least one test item was written to measure each behavioral objective. A committee composed of seven specialists in this subject independently checked each test item to determine if the Stem actually measured the behavior and to determine the test item's cognitive level (knowledge, comprehension, and application). Test items were selected for which there was at least 30% agreement. Finally, the first and second formative subtests were combined together as one test, and the third, fourth, and fifth subtests were combined as another test that was administered to a pilot group. Reliability coefficients for the formative tests were as follows: .78 for the first, .SI for the second. .80 for the third. .79 for the fourth, and .84 for die fifth.

     The Summative Test—Curriculum Development and Instruction (CDI) Test—was developed by the same procedure explained above and was administered as a pretest and a posttest. This test contained 80 items that assessed the knowledge, comprehension, and application levels ot each learning task. The reliability coefficient of this test was .85 (K-R 20).

     Treatments

     All participants received the same teaching except for the independent variables under investigation, that is enhanced cognitive entry behavior and feedback/corrective procedures given, by formative tests.

     In the control group, after the Cognitive Entry Behavior Test and the Summative Test were administered as pretests, a conventional method was used. Students learned in a class with about 30 students per teacher. They were given the course objectives, an outline of content (syllabus), and a reading list at the beginning of the course and before each learning task. Mainly lectures, but also seminars and work-shops. were used as teaching methods. Formative tests were administered at the end of each learning task, but solely to determine students' marks. Students received each test score, but not data on how any lack of learning related to the behavioral objectives. Finally, at the end of the term, the summative test was administered as a posttest. Students' course grades were determined by their performance on the formative tests (30%) and by the summative test (70%).

     In Experimental Group 1 (CEB group), at the beginning of the course, gaps in prerequisite learning of each student were determined by administering the CEB Test. Items achieved by fewer than 70% of students were retaught by teachers. Other items were learned by small groups of students helping each other under the guidance of a teacher. The few students who needed further knowledge were given a reading list and a home assignment, and their preparation was checked outside the classroom. About 9 hr were spent during the first 3 weeks of the course on these enhancing procedures. After the students completed this process, they were given the Cognitive Entry Behavior Test to assess whether they had mastered the prerequisites. For the remainder of the course, this group was taught by conventional methods, and formative tests and the CDI Test were administered to determine students' marks, as described for the control group.

     In Experimental Group 2 (CEB+FC group), the students not only received enhancement in the prerequisites they lacked, as described for Experimental Group I. but also were provided with feedback and corrections. This procedure is very similar to the method of enhancing prerequisites. To remediate lack of learning in each learning task, we first analyzed the results of the formative test to determine the achievement of every student on each item. Each student was then told what she or he had failed to learn and how to remedy this situation. Second, if the majority of students had not learned the behavior, the teacher remediated using different kinds of cues that were more appropriate to the students' learning. If a small group of students had-not. learned the items, students helped each other in the classroom under the guidance of the teacher. If few students needed help, a suitable reading list and a home assignment were given. Students' course grades were determined by formative and summative tests, as described for the control and first experimental group.

     This study was conducted over the 3,5-month spring term. After completing this course, all the subjects took the summative test as a posttest.

     Results

     Means and standard deviations for each of the three groups on the CEB pretest are given in Table 1. In addition, results on the CDI pretest can be found in the last half of Table 3. Prior differences were extremely small, and one-way analysis of variance confirmed that they were not significant at the 5% level.

     Before examining the findings related to differences in outcomes among the groups, we need to examine the findings demonstrating that the conditions of the experiment were met As the prerequisites were remediated, we expected that the correlation coefficients between the CEB test score and the CDI posttest score would be significantly lower than the correlation between the CEB test score and. the CDI pretest score. In other words, when entry characteristics are enhanced, the level of prediction of the CEB Test score on the CDI score should be reduced significantly.

     Although, at the beginning, the correlations were significant between total CEB. CEB subtest scores, and CDI pretest scores in all of the groups, after cognitive entry behaviors were enhanced in the experimental groups, the correlations between total CEB, CEB subtests scores, and CDI posttest scores were substantially decreased (see Table 2). In contrast, some correlations increased in the control group. This indicates that the conditions of the experimental design were met and the prerequisites determined by the specialists were suitable for the CDI course.

     The main findings related to the idea outlined in the introduction are summarized in Table 3, where CDI pre-and posttest scores are reported for each of the three groups. An analysis of covariance (ANCOVA) was used to test for differences in achievement gains among the three groups (Kirk. 1968. pp. 455—1-72). The posttest scores were used as the dependent variable and the pretest scores were used as a covariate to statistically control for initial differences in achievement in the subject area considered. Substantial improvement in test scores occurred for the controll group and both experimental groups, but an analysis of covariance confirmed significant differences among them. F(2. 86) = 6.0.1, p< .01.

Table 1. Means and Standard Deviations on Pretest and Posttest Administration of the Cognitive Entry Behavior (CEB) Test — Total Scores and Subtest Scores

     Tukey's tests, performed on the adjusted posttest means indicated that the achievement of the second experimental group (CEB+HC) was significantly higher than mat of the control group (q = 6.770, p < .01) and of the first experimental group (CEB) (q - 3.099. p < .05) . Level of learning in the first experimental group was also significantly higher than that of the control group (q = 3.51. p < .05).

     Table 3 also contains the results, by cognitive domain, of the first and second administration of the CDI test. ANCO-VA results suggest that there was no significant difference in the level of learning among the three groups with regard to level of knowledge, F(2, 86) = 2.32. p > .05, bur significant differences were found in relation to level of comprehension. F(2. 86) - 6.32, p < .01. and application, F(2. 86) = 3.15, p < .05. Tukey's tests, performed on the adjusted means of the level of learning at comprehension, revealed that the CEB+FC group was significantly higher than the control group (q = 6.97. p < .01) and the CEB group (q = 3.41,p< .05). Additionally, at this level, achievement of the CEB group was higher than that of the control group (q -3.30. p<. 05).

     The results from applying Tukey's tests to the adjusted means of the application level indicated that the CEB+FC group was significantly superior to the control group (q — 3.57,p< .05) and the CEB group (q = 2.91.p< .05). However, there were no significant differences between the CEB and control groups in terms of achievement of the application level (q = .82. p> .05).

DISCUSSION

     The Curriculum Development and Instruction course, on which this study was based involves a less sequential subject—that is each learning task can help one to learn later tasks, but lack of learning on the previous task does not definitely prevent the learning of the next task. The relevant cognitive entry behaviors for enhancement are also not pan of the subsequent subject, as they were in Leyton's (1983) and Sayar's (1986) studies. Despite, this, enhancing cognitive entry behavior alone increased significantly the level of learning of the first experimental group as compared with the control group.

     This result indicates that, even at the undergraduate level and in a less sequential subject series, enhancing initial prerequisites alone has a positive effect on the level of learning. Thus, enhancing cognitive entry behaviors can be part of the strategy for tackling the "two sigma problem" even in such subjects and at the university level.

Table 2. - In the experimental and Control Groups. Correlation Coefficients Between Total Scores of Cognitive Entry Behavior (CEB), CEB Subtest Scores and Curriculum Development and Instruction (CDI) Pretest and Posttest Scores

Table 3. Means and Standard Deviations on Pretest and Posttest Administration of the Curriculum Development and Instruction Test — Total Scores and Cognitive Domain Subscores

     The additional use of feedback/corrective procedures significantly increased the level of learning of the second experimental group relative to the first experimental and control groups. Thus, the combination of enhancing cognitive entry behaviors and feedback/corrective procedures affected the level of learning positively more than did enhancing initial prerequisites alone. That is, our results support the hypothesis that alterable variables used together contribute more to levels of learning than either of them alone, and their separate effects tend to be additive. Therefore, levels of learning can be increased more economically by such a combination than by one-to-one tutoring (Bloom, 1984; Leyton. 1983: Sayar. 1985).

     Leyton .(1983) found that the average effect of initial enhancement of prerequisites alone was about 0.6 sigma: Sayar (1986) found it to be 0.7 sigma. In this study, the average effect of initial enhancement of prerequisites was 0.4 sigma. This contrast can be explained by the characteristics of the cognitive entry behaviors. As mentioned before. in this study they were less crucial for new learning tasks than they were in studies of a more sequential subject series. The average effect of combining the two variables of initial enhancement and feedback/corrective procedures was 2.76 sigmas in Sayar's (1986) study and 1.6 sigmas in Leyton's (1983) study. The comparable figure in this study was 0.8 sigma. This lower effect is again likely the result of the less sequential nature of the course and its link with the cognitive prerequisites. Moreover, this study was carried out at the university level and university students are expected to learn more independently (Mueller, 1976). In spite of these constraints, the results indicate that when alterable variables are used together, their contribution to the level of learning is greater than enhanced CEB alone, in less sequential subjects and at the bachelor of education degree level.

     There was no significant difference between control and experimental groups in terms of level of learning at the level of knowledge of the cognitive domain. Knowledge level in the cognitive domain includes low cognitive processes such as recognition and recall. The results suggest that these kinds of behavior can be learned under almost any quality of instruction by students at the undergraduate level. An earlier study using similar feedback/corrective procedures found that this procedure did not make significant differences in level of knowledge (Aksu, 1981).

     However, both enhancing cognitive entry behaviors and .the combination of enhancing CEB and providing feedback/corrective procedures significantly increased levels of learning in the levels of comprehension and application. This finding supports the results of prior research in which behaviors related to higher cognitive process were more affected by the quality of instruction (Bloom, 1976, pp. 128-134; Lysakowsky & Walberg, 1982; Mevarech, 1985). In summary, the results support the idea that when a combination of alterable variables is used, even for a less sequential subject series and a less sequential learning task series at the university level, level of learning, particularly in higher cognitive behaviors, is significantly increased relative to level of learning with conventional methods, and the effects tend to be cumulative. Therefore, it seems that using a combination of alterable variables effectively in the teaching-learning process would help to solve the "two sigma problem" in less sequential subjects and at the university level.