Testing Conceptual Understanding in Physics Using a Browser-based Computer Managed SystemAlexander Mazzolini
School of Biophysical Sciences and Electrical Engineering, Swinburne University of Technology, Australia
Testing Conceptual Understanding in Physics (TCUP) is a CUTSD-funded project which aims to monitor the conceptual understanding of first-year university Physics students. It is designed not as an educational research tool but rather to provide timely feedback on student conceptual understanding. This feedback is available both to students and their instructors. The TCUP tests will help promote more effective teaching, and also help to evaluate the effectiveness of physics courses in teaching basic physics concepts. To promote effective learning, individual students will receive formative feedback regarding their level of conceptual understanding in each key area of their introductory physics course.
Each test (which is not intended to count towards summative assessment) is administered via an easy to operate browser-based computer managed testing system. The system, which is fully automated, also provides statistical information to the instructor regarding the performance of the class.
The TCUP project team
Project leaders: Dr. Alex Mazzolini, Dr. Margaret Mazzolini (Swinburne University of Technology) and Dr. John Humble (University of Tasmania).
Many university physics teaching staff are often dismayed at how little their students have learned after their first year of physics instruction. A number of physics education researchers, including Redish and Steinberg1, have suggested that many traditional teaching methods do little to improve students' conceptual understanding. Even worse, tertiary teaching staff are often unaware that a problem exists with their teaching methods. Many concerned and dedicated teachers listen to students but fail to recognise their real difficulties. Here is an example taken from Redish's recent paper: "...in a junior level electronics class, a student asked a question about a comparison of currents at two points on a single branch of a relatively complicated circuit...[The teacher]...listened carefully and recognized that the student was confused. He proceeded to give a detailed description of how the entire complicated circuit worked. However, because the current was necessarily the same throughout the branch of the circuit, it was likely that the student's difficulty was a deep one - and not addressed by the instructor's response; namely, that somehow the current was 'used up' en route [according to the student]. The student politely nodded, no better off than before, and the teacher moved on.". The above experience is not an isolated case but rather an example of what is probably a common occurrence in many university courses throughout Australia.
Thornton and Sokoloff2 have quantitatively shown that traditional teaching methods do little to improve the conceptual understanding of students undertaking introductory physics units, and that more student-centred methods of teaching need to be considered. In their study, "fewer than 10% of the students seemed to change their (incorrect) views of dynamics after traditional instruction".
Many tertiary physics academics may not be aware of the extent of the problem for the following reasons:
(i) During a physics course, students are typically given a great deal of practice in problem-solving skills via problem sheets and tutorial exercises. Students concentrate on developing these skills rather than developing conceptual understanding, since problem solving skills are the ones most likely to be tested in the final examination.
The development of the TCUP project, which tried to address the above problems, was based on three assumptions:
(i) that assessing students' conceptual knowledge and preconceptions would be a significant aid to effective teaching;
The TCUP project
TCUP is a CUTSD (Committee for University Teaching and Staff Development) funded project that has assembled an extensive set of question banks designed to test for conceptual understanding in all major areas of an introductory tertiary physics course. The multiple choice tests are delivered electronically via a browser-based computer managed testing package. The computer delivery method provides immediate feedback to students during their studies with minimal commitment of staff time. In the year 2000, the TCUP tests will be available for use in all physics departments at universities throughout Australia. The TCUP project is fully-funded by CUTSD and Swinburne University of Technology; there are no additional charges to the users of the TCUP tests.
TCUP has several strengths:
TCUP also has several limitations:
TCUP question banks
Prior to TCUP, no single, comprehensive set of question banks designed to test conceptual understanding existed across the core introductory Physics courses. Suitable question sets for parts of the core introductory Physics course (as commonly taught in Australian universities) have been developed in other places, especially the United States (e.g. Force Concept Inventory3, Mechanics Base Line Test4). Other question sets, such as the Galileo ConcepTests5, are extensive in topic coverage but have only small question banks for each topic. The TCUP project undertook an extensive survey of these and other question banks that could be used to test conceptual understanding in physics. The TCUP question banks used existing question sets as much as possible, modifying and supplementing them where necessary to cover a typical Australian introductory syllabus.
Currently, 45 TCUP tests covering electricity, magnetism, electronics, linear mechanics, rotational mechanics, waves, optics, thermal, and nuclear physics have been developed. This extensive list is shown in Table 1. The TCUP Home page is: http://www.swin.edu.au/bsee/mazzo/tcup/
It should be emphasised that the TCUP tests are not intended to form part of a graded student assessment - rather, they are a tool to aid concept development in Physics. Students will evaluate their individual progress via electronic feedback from the tests. The feedback will help them identify which particular key physics concepts require more attention in order for them to master the course material.
Table 1. Description of TCUP tests that are currently available
Electronic delivery of TCUP tests
TCUP tests are accessed from the World Wide Web via any standard web browser. Students will find the electronic delivery system easy to use, and academic staff will find it easy to administer. Figure 1 shows the operational procedure used to enable and run the TCUP tests. The browser-based, computer managed testing system being used is called WWWAssign and is written in the Perl programming language. WWWAssign was originally written by Prof. Larry Martin from North Park University and modified by Prof. Aaron Titus from North Carolina State University. TCUP has been given permission to use the shareware version of WWWAssign free of charge. It should be pointed out that a full commercial version with a robust database structure and a more sophisticated instructor interface (WebAssign) is also available for academics interested in other computer managed testing applications.
Sample Test (Instructors or students)
Standard TCUP Test (Instructors)
Standard TCUP Test (Students)
Figure 1. Operational procedures for instructors and students
To maintain test bank security, all TCUP tests must be supervised. Teaching staff wishing to use TCUP tests for their students will need to electronically import their class list and enable student access to individual tests. To do this academics must obtain an access password from the TCUP administrator. All results are recorded and processed automatically and immediately to minimise the amount of test administration for staff. The TCUP Instructors' site, which is used to enable tests, create class lists etc., is shown in Figure 2.
Figure 2. Instructors' TCUP administration page
Each test has about 10 to 15 multiple choice questions. The questions may include diagrams, and there is no time limit on completing a particular test. A typical sample test (covering electricity and magnetism) and instructions on how to access the test are available on the web site and can be used with unrestricted access. An example of a typical question taken from the sample test is shown in Figure 3. The sample test can be accessed from the TCUP home page address by clicking on the "Try out a sample TCUP Test" hot link. Other tests are also available, but require an access password.
Figure 3. Typical question from one of the "sample test" TCUP question banks
Over the past decades in Australia there have been considerable changes both in senior high school physics curricula and in the proportion of secondary students continuing to tertiary education, and consequential changes in the skills and deficiencies of students going on to study physics at the tertiary level. Many university physics teaching approaches have not adapted to accommodate these fundamental changes. It is hoped that the TCUP project will be widely used in Australian universities to provide instructors with feedback on student's levels of conceptual understanding, and to provide students with timely feedback on their level of mastery of key concepts. It is also hoped that TCUP will provide statistical data that will assist instructors to trial new teaching methods to enhance their students' conceptual understanding in physics.
CAL-laborate Volume 3 October 1999
Page Maintained By: PhySciCH@mail.usyd.edu.au