CAL-laborate Volume 3 October 1999










*******

Testing Conceptual Understanding in Physics Using a Browser-based Computer Managed System

Alexander Mazzolini
School of Biophysical Sciences and Electrical Engineering, Swinburne University of Technology, Australia

Abstract

Testing Conceptual Understanding in Physics (TCUP) is a CUTSD-funded project which aims to monitor the conceptual understanding of first-year university Physics students. It is designed not as an educational research tool but rather to provide timely feedback on student conceptual understanding. This feedback is available both to students and their instructors. The TCUP tests will help promote more effective teaching, and also help to evaluate the effectiveness of physics courses in teaching basic physics concepts. To promote effective learning, individual students will receive formative feedback regarding their level of conceptual understanding in each key area of their introductory physics course.

Each test (which is not intended to count towards summative assessment) is administered via an easy to operate browser-based computer managed testing system. The system, which is fully automated, also provides statistical information to the instructor regarding the performance of the class.

The TCUP project team

Project leaders: Dr. Alex Mazzolini, Dr. Margaret Mazzolini (Swinburne University of Technology) and Dr. John Humble (University of Tasmania).
Question Bank developer: Prof. Bill Rachinger (Monash University and Swinburne University of Technology).
Web delivery: Dr. Barbara Moss (Swinburne University of Technology).

Introduction

Many university physics teaching staff are often dismayed at how little their students have learned after their first year of physics instruction. A number of physics education researchers, including Redish and Steinberg1, have suggested that many traditional teaching methods do little to improve students' conceptual understanding. Even worse, tertiary teaching staff are often unaware that a problem exists with their teaching methods. Many concerned and dedicated teachers listen to students but fail to recognise their real difficulties. Here is an example taken from Redish's recent paper: "...in a junior level electronics class, a student asked a question about a comparison of currents at two points on a single branch of a relatively complicated circuit...[The teacher]...listened carefully and recognized that the student was confused. He proceeded to give a detailed description of how the entire complicated circuit worked. However, because the current was necessarily the same throughout the branch of the circuit, it was likely that the student's difficulty was a deep one - and not addressed by the instructor's response; namely, that somehow the current was 'used up' en route [according to the student]. The student politely nodded, no better off than before, and the teacher moved on.". The above experience is not an isolated case but rather an example of what is probably a common occurrence in many university courses throughout Australia.

Thornton and Sokoloff2 have quantitatively shown that traditional teaching methods do little to improve the conceptual understanding of students undertaking introductory physics units, and that more student-centred methods of teaching need to be considered. In their study, "fewer than 10% of the students seemed to change their (incorrect) views of dynamics after traditional instruction".

Many tertiary physics academics may not be aware of the extent of the problem for the following reasons:

(i) During a physics course, students are typically given a great deal of practice in problem-solving skills via problem sheets and tutorial exercises. Students concentrate on developing these skills rather than developing conceptual understanding, since problem solving skills are the ones most likely to be tested in the final examination.
(ii) Often there is subtle pressure on academics to set predictable examinations. Unconventional questions invariably lead to poor examination results and pass rate problems that require justification to Heads of Department or Deans. As a consequence, students learn how to answer the predictable question styles and can often pass examinations without a good understanding of the material presented in the course.

The development of the TCUP project, which tried to address the above problems, was based on three assumptions:

(i) that assessing students' conceptual knowledge and preconceptions would be a significant aid to effective teaching;
(ii) that concept development is a crucial learning outcome not sufficiently catered for in standard physics courses; and
(iii) that physics departments need tools to evaluate the success of their courses, if they are to be persuaded to trial alternative methods of instruction.

The TCUP project

TCUP is a CUTSD (Committee for University Teaching and Staff Development) funded project that has assembled an extensive set of question banks designed to test for conceptual understanding in all major areas of an introductory tertiary physics course. The multiple choice tests are delivered electronically via a browser-based computer managed testing package. The computer delivery method provides immediate feedback to students during their studies with minimal commitment of staff time. In the year 2000, the TCUP tests will be available for use in all physics departments at universities throughout Australia. The TCUP project is fully-funded by CUTSD and Swinburne University of Technology; there are no additional charges to the users of the TCUP tests.

TCUP has several strengths:

  • Course coordinators will be able to provide regular diagnostic feedback to encourage students to improve their understanding of fundamental concepts in their physics courses.
  • The process of "working through" the TCUP question banks will in itself signal to students that a deeper approach to learning is a requirement for conceptual understanding, and will show that conceptual understanding as well as problem solving skills are necessary outcomes of an introductory physics course.
  • Teaching staff will also receive immediate, automatically generated diagnostic feedback on how well each student group (as a whole) mastered key concepts. This will allow the staff the option of undertaking remedial action and adjusting their teaching emphasis accordingly.
  • The TCUP question banks will allow teaching staff to do pre- and post- course concept-mastery testing. This will aid individual staff in judging the degree by which their existing courses are successful in transferring basic physics concepts. It will also be an invaluable tool in the trialling of new courses and alternative techniques of course delivery.
  • TCUP also has several limitations:

  • Ideally TCUP questions and choice of multiple choice distractors would be based on open-ended questions, preferably in the context of student interviews. However this is a time-consuming process, and diagnostic testing as a feedback tool will only be effective if it covers most of the "standard" syllabus. As TCUP aims to provide a mechanism for useful feedback rather than to be an educational research tool, it was decided instead to rely on individual expertise, plus pre-existing diagnostic tests where available, (in particular, the expertise of Prof. Bill Rachinger, who has developed diagnostic tests in physics over many years, and his surveys of the literature reporting student interviews).
  • The TCUP tests are purely multiple choice format. The WWWAssign computer testing system used to deliver the tests can also use numeric and other formats. However, given that the TCUP project required automatic marking of student answers (which ruled out extended answer format) and that TCUP is testing conceptual understanding rather than problem solving techniques (which reduces the need for numeric answer format), it was decided that TCUP questions would be restricted to multiple choice format. Multiple choice, a much-discussed and often maligned testing format, does have the advantage that feedback can be directed specifically at "distractors" which match common student misconceptions.
  • TCUP question banks

    Prior to TCUP, no single, comprehensive set of question banks designed to test conceptual understanding existed across the core introductory Physics courses. Suitable question sets for parts of the core introductory Physics course (as commonly taught in Australian universities) have been developed in other places, especially the United States (e.g. Force Concept Inventory3, Mechanics Base Line Test4). Other question sets, such as the Galileo ConcepTests5, are extensive in topic coverage but have only small question banks for each topic. The TCUP project undertook an extensive survey of these and other question banks that could be used to test conceptual understanding in physics. The TCUP question banks used existing question sets as much as possible, modifying and supplementing them where necessary to cover a typical Australian introductory syllabus.

    Currently, 45 TCUP tests covering electricity, magnetism, electronics, linear mechanics, rotational mechanics, waves, optics, thermal, and nuclear physics have been developed. This extensive list is shown in Table 1. The TCUP Home page is: http://www.swin.edu.au/bsee/mazzo/tcup/

    It should be emphasised that the TCUP tests are not intended to form part of a graded student assessment - rather, they are a tool to aid concept development in Physics. Students will evaluate their individual progress via electronic feedback from the tests. The feedback will help them identify which particular key physics concepts require more attention in order for them to master the course material.

    Table 1. Description of TCUP tests that are currently available

    TestNameDescription
    Sample TestSample TestContains examples of EM questions from TCUP tests.
    EM1Introductory ElectrostaticsForces between electric charges.
    EM2Electric FieldThe concept of electric field as force per unit charge; field due to charge distributions.
    EM3Electric PotentialThe concept of electric potential as potential energy per unit charge; potential due to charge distributions; equipotential lines.
    EM4Electric FluxIntroduction to the concept of flux; flux through open and closed surfaces in various situations.
    EM5Gauss's Law IGauss's Law applied to simple charge distributions.
    EM6Gauss's Law IIFurther applications and consequences of Gauss's Law.
    EM7Field and Potential revisitedSituations interlinking the concepts of electric field and potential.
    EM8Conductors and InsulatorsConductors, insulators and dielectrics contrasted in their effects on field and potential.
    EM9Magnetic Fields IIntroduction to magnetism; forces on moving charges; superposition of fields.
    EM10Magnetic Fields IIMotion of charges in electric and magnetic fields.
    EM11Magnetic Fields IIIForces on current-carrying conductors in uniform magnetic fields.
    EM12Ampère and Biot-Savart LawsMagnetic fields due to electric current distributions; laws of Ampère and Biot-Savart.
    EM13Induced EMFVarying magnetic fields and induced EMF; Lenz's and Faraday's Laws.
    EM14Circuits ICircuits with batteries and bulbs.
    EM15Circuits IIOhm's Law; current and potential in single loop circuits.
    EM16Circuits IIIEnergetics of circuits; multiloop circuits.
    EM17LCR TransientsSwitching transients in circuits containing resistors, capacitors and inductors.
    EM18AC CircuitsAC circuits with LCR elements; time variation of current and voltage; phasor diagrams.
    W1Basics of 1-D WavesReflection and superposition of waves and pulses in strings.
    W2Sinusoidal WavesMathematical representation of a travelling sinusoidal wave and its interpretation.
    W3Superposition of WavesSuperposition of travelling waves; standing waves.
    MECH1Introductory MechanicsMechanics revision.
    MECH2One Dimensional KinematicsDisplacement-time and velocity-time graphs and their interpretation.
    MECH3Two Dimensional KinematicsVector displacement, velocity and acceleration; parabolic and circular motion.
    MECH4ForcesResolution and addition of forces; pulley and inclined plane situations; Newton's Third Law.
    MECH5Forces and MotionNewton's Second Law. Forces due to gravity, friction and tension. Forces in free flight and in accelerating elevators.
    MECH6Mechanical EnergyWork, kinetic energy and potential energy. Conservation of Energy.
    MECH7Linear MomentumMomentum conservation. Systems of particles. Centre of Mass. Collisions.
    MECH8Dynamics RevisionAn integrated look-back at various aspects of dynamics.
    MECH9Rotational KinematicsAngular velocity, angular acceleration and their vector representation.
    MECH10Rotational Dynamics ITorque, angular momentum, rotational kinetic energy, rotational inertia.
    MECH11Rotational Dynamics IIRotational equivalent of Newton's Second Law.
    MECH12OscillationsDynamics of the simple harmonic oscillator. Effect of damping.
    MECH13Gravitation and OrbitsNewton's Law of Gravitation. Gravitational potential. Orbital motion.
    OPT1Images and MirrorsReal and virtual images. Images formed by plane and spherical mirrors using ray tracing methods.
    OPT2Refraction and LensesRefraction at interfaces. Total internal reflection. Images formed by converging and diverging lenses using ray tracing methods.
    OPT3Interference and DiffractionTwo source interference. Edge and single slit diffraction.
    HEAT1Heat FundamentalsThe nature of heat and microscopic interpretations of thermal phenomena.
    HEAT2Specific and Latent HeatsUnderstanding and using specific and latent heats.
    HEAT3Heat Transfer ProcessesThe microscopics of heat conduction.
    HEAT4Heat ConductionHeat conduction in solids.
    HEAT5RadiationHeat transfer by radiation.
    HEAT6Elementary ThermodynamicsElementary Thermodynamics, P-V diagrams.
    NUC1Nuclear Physics IIntroductory nuclear physics.
    NUC2Nuclear Physics IIAdvanced nuclear physics.

    Electronic delivery of TCUP tests

    TCUP tests are accessed from the World Wide Web via any standard web browser. Students will find the electronic delivery system easy to use, and academic staff will find it easy to administer. Figure 1 shows the operational procedure used to enable and run the TCUP tests. The browser-based, computer managed testing system being used is called WWWAssign and is written in the Perl programming language. WWWAssign was originally written by Prof. Larry Martin from North Park University and modified by Prof. Aaron Titus from North Carolina State University. TCUP has been given permission to use the shareware version of WWWAssign free of charge. It should be pointed out that a full commercial version with a robust database structure and a more sophisticated instructor interface (WebAssign) is also available for academics interested in other computer managed testing applications.

    Sample Test (Instructors or students)
    Figure 1a.
    Standard TCUP Test (Instructors)
    Figure 1b.
    Standard TCUP Test (Students)
    Figure 1c.

    Figure 1. Operational procedures for instructors and students

    To maintain test bank security, all TCUP tests must be supervised. Teaching staff wishing to use TCUP tests for their students will need to electronically import their class list and enable student access to individual tests. To do this academics must obtain an access password from the TCUP administrator. All results are recorded and processed automatically and immediately to minimise the amount of test administration for staff. The TCUP Instructors' site, which is used to enable tests, create class lists etc., is shown in Figure 2.

    Figure 2.

    Figure 2. Instructors' TCUP administration page

    Each test has about 10 to 15 multiple choice questions. The questions may include diagrams, and there is no time limit on completing a particular test. A typical sample test (covering electricity and magnetism) and instructions on how to access the test are available on the web site and can be used with unrestricted access. An example of a typical question taken from the sample test is shown in Figure 3. The sample test can be accessed from the TCUP home page address by clicking on the "Try out a sample TCUP Test" hot link. Other tests are also available, but require an access password.

    Figure 3.

    Figure 3. Typical question from one of the "sample test" TCUP question banks

    Conclusion

    Over the past decades in Australia there have been considerable changes both in senior high school physics curricula and in the proportion of secondary students continuing to tertiary education, and consequential changes in the skills and deficiencies of students going on to study physics at the tertiary level. Many university physics teaching approaches have not adapted to accommodate these fundamental changes. It is hoped that the TCUP project will be widely used in Australian universities to provide instructors with feedback on student's levels of conceptual understanding, and to provide students with timely feedback on their level of mastery of key concepts. It is also hoped that TCUP will provide statistical data that will assist instructors to trial new teaching methods to enhance their students' conceptual understanding in physics.

    References

    1. Redish, E. F. and Steinberg, R. N. (1999) Teaching Physics: Figuring Out What Works, Physics Today, January, 24-30.
    2. Thornton, R. K. and Sokoloff, D. R. (1998) Assessing student learning of Newton's laws: The Force and Motion Conceptual Evaluation and the Evaluation of Active Learning Laboratory and Lecture Curricula, Am. J. Phys., 66, 338-352.
    3. Hestenes, D., Wells, M. and Swackhamer, G. (1992) Force Concept Inventory, The Physics Teacher, 30, March, 141.
    4. Hestenes, D. and Wells, M. (1992) A Mechanics Baseline Test, The Physics Teacher, 30, March, 159.
    5. Mazur, E. http://galileo.harvard.edu/home.html

    Alexander Mazzolini
    School of Biophysical Sciences and Electrical Engineering
    Swinburne University of Technology
    PO Box 218
    Hawthorn
    Victoria 3122
    Australia
    amazzolini@swin.edu.au


    Return to Contents

    CAL-laborate Volume 3 October 1999

    [an error occurred while processing this directive]

    Page Maintained By: PhySciCH@mail.usyd.edu.au
    Last Update: Monday, 30-Apr-2012 15:15:37 AEST
    URL: http://science.uniserve.edu.au/pubs/callab/vol3/mazzo.html