The effectiveness of the Thai traditional teaching in the introductory physics course: A comparison with the US and Australian approachesNarumon Emarat, Kwan Arayathanitkul, Chernchok Soankwan and Ratchapak Chitaree
Department of Physics, Faculty of Science, Mahidol University, Rama 6 Road, Bangkok, Thailand
School of Physics, The University of Sydney, NSW 2006, Australia
This paper was first published as a refereed paper in 'Proceedings of Scholarly Inquiry in Flexible Science Teaching and Learning Symposium, April 5, 2002' and is reprinted here with the express permission of the authors.
In recent years, a substantial and growing body of research in physics education has been involved with identification of student misconceptions especially in the fundamental physics. Misconceptions are ideas or concepts that students have developed, based on their own experiences, which are often in conflict with the physics point of view. For example, many students believe that if an object is in motion, there must be a force acting on it. It is commonly accepted among researchers in this field that such students have failed to develop a Newtonian way of thinking about mechanics, which is the view held by the physics community. (A collection of the important papers in this field can be found in Pfundt and Duit (1994).) Researchers have shown that misconceptions are widely shared, the same ones appearing again and again in different groups of students. They have also shown that traditional instruction is relatively ineffective in correcting these misconceptions or in helping students develop a more 'appropriate' way of thinking (see for example, McDermott (1990)).
In the last decade or so, much work has been done on developing special diagnostic tests to uncover misconceptions and to investigate students' understanding of physics concepts - see for example, Hestenes (1998). These tests usually consist of multiple choice questions in which the correct answer is hidden among very attractive wrong answers. These wrong answers are, in fact, constructed from common misconceptions identified by earlier researchers. Among the best known of the physics tests in the area of dynamics and kinematics are: the Force Concept Inventory (Hestenes, Wells and Swackhamer 1992); the Test for Understanding Graphs in Kinematics (Beichner 1994); and the Force and Motion Conceptual Evaluation (FMCE), designed by Sokoloff and Thornton (1998). Much effort within the Physics Education Research community has gone into evaluating these tests, both by themselves and in relation to one another (see for example, Huffman and Heller (1995)).
Administration of these standardized tests to many groups of students (mostly within the USA) has led researchers to the conclusion that (1) in general, the understanding of concepts in mechanics by introductory physics students is quite poor, and (2) that this low level of understanding is not much improved by the standard teaching given in most universities - so long as the teaching is 'traditional', i.e. consists mainly of lectures and laboratories. On the other hand, where innovative teaching methods, usually referred to as 'interactive-engagement' methods, are used, considerable gains can be achieved. For a definitive review of all these findings see Hake (1998).
The current authors are interested in whether these same general findings can be extrapolated to other cultures, or whether they are only really applicable within the USA. We focus attention on one of the above standardized tests, the FMCE, because the originators of that test have also developed a particular interactive-engagement teaching technique which targets the same concepts as the test addresses. Reports of the testing of their own students can be briefly summarized thus. (1) The great majority of these students entered a university without a correct, or Newtonian, point of view on kinematics and dynamics, and (2) after instruction by the new teaching method, some 80-90% of their students were able to complete the FMCE successfully (a much higher fraction than in parallel, traditionally taught classes). See Sokoloff and Thornton (1997) for details. Some teachers in other institutions have used the same methods and report similar results (Cummings et al. 1999).
In Australia, Johnston and Millar (2000) did the same experiment and found comparable results, with one major difference. When the test was administered to introductory physics students before any instruction had taken place, the students' understanding of the concepts (as measured by the FMCE) was markedly higher than for US students. Since the universities involved in all these trials seemed to be much the same as regards entrance requirements and so on, this finding is interesting, though its significance is not clear.
For many reasons therefore, it would seem important to ask whether these findings are valid only for Western educational systems, or whether they are also likely to apply to, for example, the educational system in South East Asia. As a first step in answering this question it was decided to test students in a non-Western context in order to study their pre-university level of understanding and the effectiveness of traditional teaching. The same FMCE test was given, before and after instruction, to 1300 physics first year students at Mahidol University in Bangkok, Thailand.
The Force and Motion Conceptual Evaluation (FMCE)
The FMCE is a research-based multiple choice assessment instrument that was designed to probe a conceptual understanding of Newtonian mechanics. It consists of 43 questions, which are divided into 8 sets. Thornton and Sokoloff (1998) focused on the following four sets of the test.
Set 1: Natural Language Evaluation (questions 1, 2, 3, 4 and 7). This set consists of five force-sled questions, asking students to relate a force to various motions of the sled. All the questions make no reference to graph or coordinate system. The questions in this set are as follows.
'Choose the force which would keep the sled moving as described.
Set 2: Graphical Evaluation (questions 14, 16, 17, 18, 19, 20 and 21). This set uses graphical representation in the answers and does not explicitly describe the force that is acting on an object. All questions are asked in the same way as those in set 1, so they measure the same concepts in physics.
Set 3: Coin Toss (questions 11, 12 and 13). This set of three questions asks students to select a force acting on the coin tossed straight up into the air. The questions in this set are as follows.
'Indicate the force acting on the coin for each of the cases described below.
Set 4: Cart on Ramp (questions 8, 9 and 10). This set of three questions is similar to set 3 except that the situation is changed from a coin tossed into the air to a cart pushed and released up the ramp.
Full detail of the test as well as deeper discussion and analysis of the test can be seen in Thornton and Sokoloff (1998).
English is not the native language of Thai people. Many Thai students have problems with English questions. Therefore, it is impossible to use the FMCE test with Thai students without translation. The translation was carefully done by an experienced Thai physics professor at Mahidol University. He has done many translations of English physics problems into Thai ones. The Thai version of the test uses technical terms understandable by first year students. Each question was translated in the way that all its original meanings are kept and no further explanations are given. The translation into Thai was validated by 20 academic staff and graduate students in the physics department at Mahidol University. They were asked to do both Thai and English versions of the test. The Thai test was given first and then the English test. Therefore, the staff and students had no chance to translate the test on their own. With minor adjustment of the translation, all of the staff and students arrived at the same answers for each question in both Thai and English tests. In other words, if a person made a mistake in one of the questions in the Thai test, he also made the same mistake in that particular question in the English test.
We have done the test at Mahidol University, which is one of the best universities in Thailand, especially in the fields of science and medical science. The first year calculus-based physics course at Mahidol University enrols around 1300 students. The students major in medical science, engineering and pure science. They were generally divided into 6 classes. Each class had roughly 200 students and was taught by different lecturers. The physics course in the first semester consists of four topics: Mechanics, Waves, Thermodynamics and Electromagnetism. The 12 hours of Mechanics take 6 weeks of lectures.
The students were asked to do the FMCE test during the first week of the first semester before the traditional instruction was given. The Mechanics lecture lasted 6 weeks and covered dynamics, kinematics, work, energy and rotation. The traditional instruction includes standard lectures, homework problems, and quizzes. The students also enrolled in a separate course of physics laboratories with weekly experiments. The students were told that test results had no effect on their grades, but they would get a few points in reward for doing the test. Three weeks after the end of the Mechanics lecture, which is also one week after the regular mid-semester examination, the students were given the same test again.
Results and discussions
The pre-test results of the experiment are shown in Figure 1 in which student responses are reported for four sets of questions.
Figure 1. The percentage of correct responses in the pre-test from 1300 students at Mahidol University
The first point to be noted is that around 40% of the students answered the dynamics questions in set 1, the natural language evaluation, in the ways that are consistent with a Newtonian view of the world. The graphical evaluation that roughly asks the same questions, yields lower percentage. This is possible owing to the lack of practice on the graphical part of dynamics for Thai students.
For the coin toss and the cart on ramp sets of questions, we follow Thornton and Sokoloff (1998) by considering that students have the Newtonian point of view only when all three questions in each set are answered correctly. The results of these two sets (see Figure 1) show that less than 20% of the students have the Newtonian point of view.
For detail of the distribution of marks on selected test items, we choose to show percentage of students getting the correct answer in each question of set 1 and set 3. This is shown in Figure 2.
Figure 2. Pre-test and post-test percentages of correct responses to questions in set 1 and set 3
Low scores in question 2 of set 1, both in pre- and post-tests, indicate students' misconception in relating force with motion even when the sled is moving at constant velocity. Relatively low scores are also found in all questions of set 3, especially question 11. The changes before and after traditional instruction averaged about 9.0% and 9.8% for set 1 and set 3 questions, respectively. Such low improvement on these questions may be due to the wrong assumption of the teacher that students have already had the right concept about force and motion before entering the university. (The teacher was not provided with the pre-test results before giving the lecture). The results also indicate that most of the students still use their own concepts and do not accept the Newtonian point of view. They somehow relate the direction of force with the direction of motion.
Figure 3 shows the student understanding before and after traditional instruction for all sets of questions. It is clear that the lecture has small effect on student understanding since the total change before and after traditional instruction is about 9.7% in average. This is quite a low gain after the traditional instruction was given although there is a big room for improvement due to the low pre-test scores.
Figure 3. Comparison of the percentage of correct responses before and after traditional instruction at Mahidol University
Comparison of the pre-test scores of Thai, Australian (Johnston and Millar 2000) and US (Thornton and Sokoloff 1998) students on the same sets of questions is shown in Figure 4. All four sets of question show the same trend. The pre-test scores of Thai students are between the US and Australian. The average of the pre-test scores for US, Thai and Australian are 9%, 27% and 49% respectively.
Figure 4. Pre-test percentage of correct responses to questions in four sets, as published by Thornton and Sokoloff (US), Johnston and Millar (Australian) and at Mahidol University (Thai)
A comparison of the gains from the three different contexts is shown in Figure 5. In all cases the student gain was quite low. In fact, the averaged gains are almost the same. They are 8.0%, 9.7%, and 10.1% for US, Thai and Australian students, respectively. The three contexts have similar gain despite the fact that their pre-test scores are quite different. These gains confirm the worldwide-accepted conclusion that traditional instruction is ineffective in teaching physics concepts and in changing misconceptions.
Figure 5. The percentage of gains after the traditional instruction on questions in four sets, as published by Thornton and Sokoloff (US), Johnston and Millar (Australian) and at Mahidol University (Thai)
We have done the conceptual evaluation test with around 1300 students in a Thai university. The result from the pre-test shows that a few students entering a university understand force and motion from the Newtonian point of view. After a semester of traditional instruction the improvement in performance is found to be quite poor. There was an increase of only 10% from the pre-test scores. Such results have also been found in universities and colleges in the US and in Australia, as reported in the literature (Thornton and Sokoloff 1998; Johnston and Millar 2000). The findings of this project therefore support the widely held view that traditional teaching is relatively ineffective in helping students to learn physics concepts and in changing misconceptions. It is also interesting that the average of the pre-test scores in the Thai context is 27% which is lower than for Australian students but higher than for US students. Again the significance of this is not clear and calls for further investigation.
We believe that it is possible to conclude that the 10% improvement points to the ineffectiveness of traditional teaching on mechanics, in Thailand as in the USA and Australia. The second stage of this project must therefore be to test whether a significant improvement in understanding (as measured by the MFCE) can be achieved by replacing traditional teaching strategies with more interactive learning ones (see, for example, Sokoloff and Thornton (1997) and Johnston and Millar (2000)).
The authors would like to thank Assistant Professor Piyapong Sithikong at the Department of Physics, Faculty of Science, Mahidol University, who helped translate the FMCE paper. The test materials and the funding for a trip to present this paper at the UniServe Science Conference 2002 are supported by the Faculty of Science, Mahidol University.
© 2002 N. Emarat, K. Arayathanitkuk, C. Soankwan, R. Chitaree and I. Johnston
Narumon Emarat, Kwan
Soankwan and Ratchapak
CAL-laborate Volume 9 October 2002
Page Maintained By: PhySciCH@mail.usyd.edu.au