An Exercise in Course EvaluationChernchok Soankwan
Mahidol University, Thailand
The University of Sydney
In April/May, 2000, one of the authors (CS) was a visiting fellow at UniServe Science and the School of Physics, The University of Sydney, under the Thailand Australia Science and Engineering Assistance Project (TASEAP). As part of that fellowship he undertook to evaluate the effectiveness of a course in computational physics which the School had just introduced. This module, an update of a long-standing component of the experimental physics curriculum for advanced students, introduced them to the topic of 'Oscillations and Chaos' by means of a newly written set of computer exercises based on the mathematical package MatLab1. The purpose of the evaluation was not only to allow the School to monitor the effectiveness of this particular item in its teaching program, but also to establish protocols for future evaluations of other parts of the curriculum.
Description of the course
The computational physics module, 'Oscillations and Chaos', is one of six in Experimental Physics: Physics 1901 (Advanced). It consists of three-hour sessions in the laboratory, which students are required to attend once a week for three weeks. They work in groups of two or three, through a set of exercises provided in the laboratory notes2. Each group has access to a computer for the entire session. Students can obtain help during the session from tutors.
As with many university courses, the primary aims of this course had never been explicitly formulated before it was put together. However consultation with the teachers and tutors reached agreement that the course's main aims were:
The project sought to gather information about the students in three areas: attitudes; knowledge; and skills (see Table 1). Three different ways of collecting information were used: before and after interviews with participating tutors and lecturers; questionnaires for evaluation of the students' performance at the beginning and end of the course; and observation of how students approached their work during class. The objectives of each method are summarized in Table 2.
Table 1. The three facets of information gathered
Table 2. The three methods and their objectives
Design of the Pre-Questionnaire
It was decided that it was important to determine the students' backgrounds with which they came to this course in two categories.
Design of the Post-Questionnaire
The post questionnaire was designed to check the development of the students in the same two areas.
Results and discussion
The raw data from all of these information-collecting exercises can be viewed on-line5. Only the most significant findings are reported here.
Students' attitudes toward their own computer skills
More than 80% of the students felt very comfortable with their basic computer skills such as using Windows, managing files, backing up data and using the mouse and keyboard. This implies that students came to this course with adequate basic computer skills. On the other hand, they lacked experience in writing programs and using mathematical packages, implying they had little idea about using the computer as a tool for solving mathematical problems.
After the students had finished this course, more than 90% agreed that computational methods are helpful in understanding physics phenomena and in solving physics problems. In addition, most agreed that MatLab is useful for tackling many mathematical problems. These results show that, at least in their own estimation, this course did give students a feeling for the role of computational methods in physics - which was one main aim of the course.
Students' attitudes toward their own mathematical and physics skills
At the start of the course most of the students were highly confident of their mathematics ability. At the end, very few felt that this course had helped in improving their mathematical skills. These findings are at least mutually consistent.
On the other hand a much smaller fraction felt they knew much of the physics at the start, particularly chaos; and at the end, more than 90% found the course to have been helpful in improving their physics knowledge. Again these results indicate success in exposing them to ideas about chaos - which was another aim of this course.
Students' attitudes toward the course as a whole
While more than 60% of the students felt that this course was interesting, enjoyable and useful in learning physics, only a very small fraction felt that they were encouraged to do further study. While not exactly contradicting any of the course aims, this feature warrants consideration in future.
Students' performance in test questions
Comparison of the results from the pre-tests and post-tests show that students did indeed improve their knowledge about oscillations; damping, driving, resonance and chaos. In addition, the number of students claiming not to know about solving differential equations decreased after they had finished the course. This has to be a success in terms of the aims.
However, a qualitative analysis of the open-ended questions indicated a marked increase in the tendency to answer those questions from a mathematical, rather than a physical, point of view. An example is the question 'What is the difference in behaviour between a pendulum swinging through a small angle and one swinging through a large angle?', which appeared on both pre- and post- tests. At the start of the course, most students answered in terms of actual behaviour - the period gets longer, or similar. At the end, the number of students answering this question in terms of linearity or the approximation involved had increased dramatically. This point also needs to be investigated more closely, in view of the fact that an aim of the course was deemed to be to appreciate the role of modelling in physical understanding.
In the modern university system, in most countries in the world, increased emphasis is being placed on improving the quality of teaching. It follows that we need to develop ways of evaluating the effectiveness of our courses which are transparent, comprehensive and not too time-consuming. The authors believe that the protocols described above are just that. They could easily be transferred to other courses. There may be doubt in the reader's mind that undue stress has been placed on students' own estimation of the outcomes of the course, rather than on more 'objective' measures. This is, however, in line with current trends in educational circles to carry out evaluations which focus on the student's, rather than the teacher's, perspective, for the reason that all learning is, in the end, mediated by students' perceptions of what they are being asked to do6.
One last point should be made however. Before any reasonable evaluation of a course can be done it must be clear what the aims of the course are. As mentioned above, many university courses do not set these down before the course is put together. Too often the aims are defined in terms of the syllabus - 'They need to know x, y and z'. If that is the case, then the first job of the evaluator will be to determine what the aims are, which is indeed what happened here.
UniServe Science News Volume 16 July 2000
Page Maintained By: PhySciCH@mail.usyd.edu.au