View of the Research on the Efficacy of CAI
Michael S. Jenks
John M. Springer
The Association for Education Communications and Technology (1977) has defined computer-assisted instruction (CAI) as a method of instruction in which the computer is used to instruct the student and where the computer contains the instruction which is designed to teach, guide, and test the student until a desired level of proficiency is attained. In this paper the authors attempt to present a contemporary view of a body of research on the effectiveness of CAI as a teaching tool, especially as it impacts academic performance. Following the definition for CAI above, this review addresses the question of whether or not CAI can be effectively utilized to teach, guide, and test students. It is presented in two parts: A general view of the research on the efficacy of CAI and an overview of issues surrounding that research.
Dalton and Hannafin (1986, January), in a study involving junior high students, found that CAI alone tended to be the most effective instructional delivery system compared to video alone and interactive video. In a large curriculum integration project involving 1st through 8th grades, Lore and Chamberlain (1988) found that a CAI integrated curriculum was effective (e.g., reached a pre-established learning benchmark) when averaged over all grades. However, some grades performed at or above the benchmark while other grades performed below. Specifically, they found that grades 3, 4, 6, and 7 met or exceeded the goal level of academic achievement; grades 1, 2, 5, and 8 did not reach the goal level (p. 4). Price (1989) conducted an attitude survey and observed student progress in a middle school science project where CAI was used as a tutorial and research tool. It was concluded that the use of CAI in this way encouraged an overall improvement in motivation and interest in the science research project. Christmann, et al. (1997) conducted a meta-analysis of the effect of CAI in secondary education for the years of 1984 to 1995. They selected only studies that were correlative, quasi-experimental, or experimental in design and concluded that CAI had a greater effect size in the 1980s than it did in the 1990s (through 1995). Their research indicated that for the 12 year period (1984 to 1995), secondary students exposed to CAI showed "higher academic achievement than 57.2% of those students exposed to traditional instruction" (p. 328).
Research from the 1980's suggests that (CAI) has had a positive effect on a variety of students (age) in the formal education environment. Wesley, Krockover and Hicks (1985, Winter), while examining the effects locus of control has on the acquisition of computer literacy, found that externally controlled preservice teachers learned better in the CAI mode than in the comparison text mode, while internally controlled preservice teachers learned equally in both modes of instruction. Danley and Baker (1988) conducted a study involving the teaching of special education 'mainstreaming' to preservice teachers using CAI and traditional lecture modes of instruction. They found that neither method proved superior to the other in a measure of content learning. In another study involving preservice teachers, Rowland (1988, April) argued that the results of the study indicate tutorial CAI should be used for teaching basic concepts and simulation CAI would be better suited to teaching the application of concepts. Further, it was suggested that if CAI is to be effective, educators should match the CAI type to the intended learning goal/s (Rowland, 1988, April).
from the 1990's also suggests that CAI has had a positive effect on
a variety of students in post-secondary settings. Sasser (1990-91, Winter)
found that the mathematics achievement of those preservice teachers
who received computer tutorial assignments was significantly higher
than those who received the textbook assignments. Tirosh, Tirosh, Graeber,
& Wilson, (1990-91) reported on a study designed to use CAI to correct
preservice teachers misconceptions about division story problems. It
was found that CAI was effective in improving performance in writing
expressions for division word problems, increasing awareness of a tendency
to divide by the smaller number, and helping teachers explicitly recognize
and correct their misconception about the relative size of the dividend
and the divisor. Ivers and Barron (1998) reported significant learning
increases when preservice teachers worked in a paired condition using
computer-based instruction that was designed for individual learning.
Glickman (2000) conducted a non-equivalent control group design study
and found that the CAI treatment group did significantly better than
the control group on concept understanding. However, there were no significant
differences between the treatment and control groups on a measure of
achievement. Huxford (1999), in a study comparing traditional instruction
modes with a CAI mode, argued that the results suggest CAI is not as
useful for instruction as previously believed. In the study, college
students from the different groups did not perform significantly different
on measures of cognitive and affective learning.
Overall, the review of individual studies involving college age students suggests that CAI is an effective mode of instruction. The meta-analyses below indicate the same conclusion.
Research from the 1980s and 1990s generally supports the efficacy of CAI in formal education environments. The above studies, while not representing an exhaustive listing of research, are offered in support of this assertion. The following meta-analyses (and a single review of past meta-analyses) also indicate that CAI is generally effective in education environments for a broad range of student ages.
Kulik, and Bangert-Drowns (1985) analyzed 28 studies that examined the
effectiveness of CAI with elementary students. Of these studies, 27
examined terminal-mainframe configurations, and only one looked at microcomputer
configuration. CAI related instruction appeared to improve student achievement
by 0.47 standard deviations, on average, over students receiving conventional
instruction. The authors calculated that a typical student scoring in
the 50th percentile with conventional instruction would score in the
68th percentile with CAI.
Research examining the effects of computer-assisted instruction has shown generally positive results on secondary students. Kulik (1983) found that computer-assisted instruction has the potential for improving student achievement scores in pre-college classes. The results of the meta-analysis showed that the average effect of CAI was to raise student achievement by approximately 0.4 standard deviations.
In 1983, Kulik, Bangert, and Williams analyzed 48 studies on the effects of computer-based teaching on secondary students in mathematics and science. Thirty-nine of these studies reported those students with computer-based teaching scored better on final examinations than did students in conventionally taught classes. The other nine studies reported students in conventionally taught classes scored better on final examinations.
support for computer-assisted instruction at the secondary level is
provided by Snowman (1995). The results of this study showed positive
effects of computer-based education on secondary students. The typical
student in a computer-based class scored at the 60th percentile while
the typical student in a traditional class scored at the 50th percentile
on final examinations.
Christmann and Badgett (2000) compared the academic achievement levels of college students who had classes that used traditional methodology with those of college students who had classes in which CAI was used as a supplement to traditional methodology. The authors compiled data from 26 studies and calculated an overall mean effect size of 0.127. When exposed to CAI, typical student achievement moved from the 50th percentile to the 55th percentile (Christmann & Badgett, 2000).
Lowe (2001) reviewed several other meta-analyses from the 1980s and 1990s and concluded each of the reviewed meta-analyses showed a small positive effect size for computer-based education (CBE; the author describes CBE as a term that includes CAI) over conventional instruction. However, Lowe (2001) states that research indicates that, where CBE and conventional instruction is delivered by the same instructor, the CBE advantage is reduced to insignificant levels; further, simulation and tutorials as supplements to conventional instruction appear to be the most effective.
Issues within the Research
reviewing the literature on CAI, several issues came to light. First,
a preponderance of the research occurred in the 1980s. Even recent meta-analyses
(e.g., Christmann & Badgett, 2000; Christmann & Badgett, 1997;
Fletcher-Flinn and Gravatt, 1995) included studies from the 1980s and
early to middle 1990s. Second, some research indicates that how CAI
is used will affect its effectiveness (e.g., Kulik & Bangert-Drowns,
1983; Kulik, Kulik & Shwalb, 1985; 1986; Lowe, 2001). Third, some
researchers have argued that the methodology of many studies comparing
CAI to conventional instruction has unfairly favored CAI and that this
is reflected in the results of meta-analytic studies (e.g., Fletcher-Flinn
and Gravatt, 1995; Lowe, 2001).
The bulk of research reported above occurred in the 1980s. Further, much of the research utilized in more recent meta-analyses also came from the 1980s. This fact may be especially important when considering differences between computer environments and computer users (learners in instructional environments) of the 1970s and 1980s and those of the late 1990s through 2002. For example, Kulik, Kulik, and Bangert-Drowns (1985) reported in their meta-analysis that the bulk of elementary students in the examined studies used mainframe-terminal configurations, rather than microcomputers. Much of the computing was text-based and non-graphical. In considering the trend of lower effect sizes for the early 1990s reported by Christmann et al. (1997), more contemporary studies, including meta-analyses, are needed to examine whether or not this trend has been maintained and to explore possible causes of such a trend.
Using CAI as a Supplement
CAI as a complete replacement for conventional teaching may seriously
weaken its effectiveness (Kulik & Bangert-Drowns, 1983). CAI is
not often used in this way, but Hartley (as cited in Kulik & Bangert-Drowns,
1983) managed to locate a few studies in which CAI totally replaced
conventional teaching. The results of these studies were considered
to be unimpressive. Based on Hartley's results, Kulik and Bangert-Drowns
(1983) warn that, "Total reliance on the computer as teacher therefore
seems to be one thing that school systems should avoid" (p. 154).
Critique of the Research
Clark (as cited in Fletcher-Flinn and Gravatt, 1995), argued that much of the apparent success of CAI in the research literature may be related to the uncontrolled effects of instructional method and the novelty of the medium. Clark, according to Fletcher-Flinn and Gravatt (1995), further argued that when looking at studies that utilized the same teacher and a longer than typical study duration, the differences between CAI and conventional instruction were minimized. Fletcher-Flinn and Gravatt (1995) attempted to directly address this critique in their meta-analysis. In isolating studies of long duration and that utilized the same materials and teacher in the treatment and control conditions, they found no beneficial effect in favor of CAI (Fletcher-Flinn & Gravatt, 1995). Lowe (2001), in reviewing various meta-analyses, supports this idea by stating that, "When instruction of CBE and conventional classroom is delivered by the same person, the learning advantage for CBE is reduced to insignificant levels" (p. 170). Fletcher-Flinn and Gravatt (1995) point out that isolating these conditions may be highlighting a potential difference between the CAI and conventional instruction of many studies. This difference is that much CAI is carefully designed with sequenced instruction following clear instructional objectives. This generally high quality CAI is then compared to conventional instruction that is often not as carefully designed. Fletcher-Flinn and Gravatt (1995) suggest that this may be a part of the difference between CAI and conventional instruction in many studies.
In view of the age of the research, and in examining the potential shortfalls of much of that research, more research should be conducted on what makes CAI effective. Does CAI fair as well in contemporary settings with contemporary learners? What subjects are most conducive to CAI? What is an effective relationship between CAI and conventional instruction?
Most of the research literature can be used to claim a superiority of CAI over conventional instruction. However, discussions about the methodologies of much of the research reduce the significance of such claims. When studies control for internal validity issues such as instructional equivalency and instructor equivalency, differences between CAI and conventional instruction appear to be insignificant.
Considering this still allows one to conclude that CAI appears to be at least as effective as conventional instruction. In answering the question, "Is CAI an effective teaching tool?" the answer should be, "Yes. But don't throw out conventional instruction for CAI." This is because the literature further suggests CAI is best delivered as a supplement to conventional instruction and effective instruction appears to be more a matter of quality of design rather than a matter of medium (see Lowe, 2001).
Michael S. Jenks, Ed.D., is Assistant Professor at Yuanpei University of Science and Technologyin Hsinchu, Taiwan. Also a teacher of English, Dr. Jenks prepares instructional multimedia training and instructional design materials for delivery to faculty. He is currently directing the development of the Yuanpei Instructional Multimedia Resource Center.
John M. Springer, Ed.D., is a Post-Doctoral Fellow at the National Information Assurance Training and Education Center at Idaho State University in Pocatello, Idaho. Dr. Springer serves as the center's instructional designer and his research interests include the examining the effects of using instructional systems design models on improving multimedia instructional systems.
Association for Educational Communications and Technology. (1977). Educational technology: Definition and glossary of terms (Vol. 1). Washington, D.C.: Association for Educational Communications and Technology.
Brophy, K. A. (1999). Is computer-assisted instruction effective in the science classroom? Masters Abstracts International, 37 (5), 1302. (UMI No. 1393859)
Brown, K. G. (2001). Using computers to deliver training: Which employees learn and why? Personnel Psychology, 54, 271-296.
Chang, J. C. (2000). A field test of CAI software: MagicTree. Masters Abstracts International, 38 (6), 1438. (UMI No. 1399856)
Christmann, E., Badgett, J., & Lucking, R. (1997). Progressive comparison of the effects of computer-assisted instruction on the academic achievement of secondary students. Journal of Research on Computing in Education, 29(4), 325-337.
Christmann, E. P., & Badgett, J. L. (2000). The comparative effectiveness of CAI on collegiate academic performance. Journal of Computing in Higher Education, 11(2), 91-103.
Coffland, D. A. (1999). Factors related to technology use by secondary geometry teachers. Unpublished Dissertation, Idaho State University, Pocatello, ID.
Dalton, D. W., & Hannafin, M. J. (1986, January). The effects of video-only, CAI only, and interactive video instructional systems on learner performance and attitude: An exploratory study. Paper presented at the Annual Convention of the Association for Educational Communications and Technology, Las Vegas, NV.
Danley, W. E., & Baker, C. (1988). Comparing a pre-service mainstreaming class taught by traditional methods with a similar class taught by computer assisted instruction. Computers in the Schools, 5(1-2), 151-255.
Dunn, C. A. (2002). An investigation of the effects of computer assisted reading instruction versus traditional reading instruction on selected high school freshmen. Unpublished Dissertation, Loyola University, Chicago.
Edwards, J., Norton, S., Taylor, S., Weiss, M., & Dusseldorp, R. (1975). How effective is CAI? A review of the research. Educational Leadership, 33(11), 147-153
Fletcher-Flinn, C. M., & Gravatt, B. (1995). The efficacy of computer assisted instruction (CAI): A meta-analysis. Journal of Educational Computing Research, 12(3), 219-242.
Glickman, C. L. (2000). The effects of computerized instruction in intermediate algebra. Dissertation Abstracts International, 61 (5), 1773A. (UMI No. 9973965)
Huxford, D. E. (1999). The Relative Efficacy of Computer Assisted Instruction. Unpublished Thesis, West Virginia University.
Ivers, K. S., & Barron, A. E. (1998). Using paired learning conditions with computer-based instruction to teach preservice teachers about telecommunications. Journal of Technology and Teacher Education, 6(2-3), 183-191.
Jenks, M. S. (2002). An examination of the effect of scrolled and paged text fields on academic performance in the delivery of a computer-assisted instruction module for teaching metrics to preservice teacher education students. Unpublished Dissertation, Idaho State University, Pocatello, ID.
Kulik, J. A. (1983, October). Effects of computer-based teaching on learners. Paper presented at the National Forum of the College Board Symposium on Computer Competency and the Curriculum, Dallas, Texas. (ERIC Document Reproduction Service No. ED 246 877)
Kulik, J. A., Bangert, R. L., & Williams, G. W. (1983). Effects of computer-based teaching on secondary school students. Journal of Educational Psychology, 75(1), 19-26
Kulik, J. A., & Bangert-Drowns, R. L. (1983). Effectiveness of technology in precollege mathematics and science teaching. Journal of Educational Technology Research, 12(2), 137-158
Kulik, J. A., Kulik, C. L. C., & Bangert-Drowns, R. L. (1985). Effectiveness of computer-based education in elementary schools. Computers in Human Behavior, 1(1), 59-74
Kulik, C. L. C., Kulik, J. A., & Shwalb, B. J. (1985, April). Effectiveness of computer-based adult education. Paper presented at the 69th Annual Meeting of the American Educational Research Association, Chicago, IL. (ERIC Document Reproduction Service No. ED 263 888)
Kulik, C. L. C., Kulik, J. A., & Shwalb, B. J. (1986). The effectiveness of computer-based adult education: A meta-analysis. Journal of Educational Computing Research, 2(2), 235-252
Lore, R., & Chamberlain, E. (1988). Language development component, compensatory language experiences and reading program 1987-88. Final evaluation report. Final evaluation report for Columbus Public Schools, OH, Department of Evaluation Services.
Lowe, J. (2001). Computer-based education: Is it a panacea? Journal of Research on Technology in Education, 34(2), 163-171.
Matejczyk, M. (1996). Using multimedia technology to teach 7th grade process. Curriculum Administrator, 31(2), 6-9.
Passerini, K. (2000). A comparative analysis of performance and behavioral outcomes in different modes of technology-based learning. Unpublished Dissertation, George Washington University, Washington D.C.
Price, S. M. (1989). Using CAI to improve participation and achievement in science research projects in middle school science. Unpublished M.S. Practicum Paper, Nova University.
Ravaglia, R., Suppes, P., Stillinger, C., & Alper, T. M. (1995). Computer-based mathematics and physics for gifted students. Gifted Child Quarterly, 39(1), 7-13.
Roberts, V. A., & Madhere, S. (1990). Chapter I resource laboratory program for computer assisted instruction (CAI) 1989-1990. Evaluation report. Evaluation report for the District of Columbia Public Schools, Washington, DC.
Rowland, P. M. (1988, April). The effect of mode of CAI and individual learning differences on the understanding of concept relationships. Paper presented at the Annual Meeting of the International Association for Computing in Education, New Orleans, LA.
Ryan, A. W. (1991). Meta-analysis of achievement effects in microcomputer applications in elementary schools. Educational Administration Quarterly, 27(2), 161-184
Sasser, J. E. (1990-91, Winter). The effect of using computer tutorials as homework assignements on the mathematics achievement of elementary education majors. Journal of Computers in Mathematics and Science Teaching, 10(2), 95-102.
Springer, J. M. (2002). The formative evaluation of a computer-assisted instruction module for metric area instruction, for preservice teachers: Its effect on student achievement and its congruence with the ADDIE instructional design model. Unpublished Dissertation, Idaho State University, Pocatello, ID.
Snowman, J. (1995). Computer-based education: More hype than help? Mid-Western Educational Researcher, 8(1), 32-35
Stern, R., & Repa, J. T. (2000). A study of the efficacy of computerized skill building for adolescents: Reducing aggression and increasing pro-social behavior. Research Report available from EDRS (ERIC) (http://orders.edrs.com/members/sp.cfm?AN=ED447792): New York City Board of Education.
Tirosh, D., Tirosh, C., Graeber, A., & Wilson, J. (1990-91). Computer-based intervention to correct preservice teachers' misconceptions about the operations of division. Journal of Computers in Math and Science Teaching, 10(2), 71-78.
Tseng, H.-C. (1999). Computer-assisted instruction in the Math Family. Masters Abstracts International, 37 (5), 1303. (UMI No. 1394146)
Wesley, B. E., Krockover, G. H., & Hicks, C. R. (1985, Winter). Locus of control and acquisition of computer literacy. Journal of Computer-Based Instruction, 12(1), 12-16.
[ Copyright © 2002 College of Education, Idaho State University | ISBN 0-9718446-0-7 ]
Please report any problems you may have with the site to our webmaster via email.