Student Multi-Choice Exam Revision Using the InternetJohn L Bähr and Bernard T Darnton
Physics Department, University of Otago, NZ
The Physics Department of the University of Otago has for many years, assisted students in their exam revision by supplying past exam papers for 100-level (first year) exams. A list of the correct answers to the multiple-choice section of these exams has also been provided, but these only state which of the multiple choices is the correct answer. The students are not shown why it is the correct answer, unless they seek help from a staff member. A number of years ago some slide/audio-tape sets were produced, and more recently, exam questions and answers (for 1987-1990) were put onto HyperText Stacks, but the availability (in place and time) was too severely limited to be of use to the majority of students. As first year student numbers now exceed 1000, a more accessible and efficient method of delivering the exam papers to students was required.
We are now using the Internet, to make past exam questions and answers (with working and explanation) widely accessible to students - from anywhere, at anytime, and from any computer platform. With this method we can also gather useful statistical information relatively easily about the process. This paper describes the setting up of such web pages, and how we provide students with help with their physics revision, using the method of multi-choice exams from previous years together with model answers. Statistical information on question access, and results from a student survey are also presented.
Background and history
Why do exam revision?
The learning of physics is enhanced by solving problems, and one good source of problems is past exam papers, especially multiple-choice exam questions. The Physics Department of the University of Otago has for many years, assisted students in their exam revision by supplying copies of past exam papers for 100-level (first year) exams. A list of the correct answers to the multiple-choice section of these exams has always been provided, but these only state which one of the five multiple choices is the correct answer. In order to enhance their learning from past exam questions, and to relieve the pressure on staff answering the same questions over and over again to queues of students at exam time, we have ported the exam questions to the Internet. Thus students can sit at a computer in one of 15 locations on campus, bring up the appropriate web page using a suitable browser, and 'sit' the multiple-choice section of a past exam paper by clicking on the hypertext-links. This use of the Internet makes past exam questions and answers widely accessible to students - anywhere, anytime, and from any computer platform.
First Year paper structure at Otago
The number and type of physics papers offered to first year students at the University of Otago has changed many times in the last twenty years. The present structure (see Table 1) divides students into 'physical science majors' and 'health science majors', and consists of two papers (PHSI 131 and 132) in each semester for the physical science students and two full year papers (PHSI 110 and PHSI 120) for the health science students. (The latter have lectures at half the rate of the former).
The physical science papers assume an understanding of Bursary (final year of high school) physics and mathematics. The health science papers do not assume an understanding of high school physics, and so the basic mathematics required (non-calculus mathematics) is incorporated in the paper.
The exam for each paper consists of two sections:
The use of HyperText Markup Language
The 'de facto' standard for documents on the Internet is HTML. A standard 'web page' is written in this language, which is relatively easy to use, and there are many web page editors available for this. The creation of physics web pages must meet a number of criteria, including the ability to produce superscript/subscript, Greek letters, equations, and diagrams.
Description of the project
General form of the pages
Figure 1 shows the opening page for the exams. From here the students select, via the two 'pull-down' menus, the paper and year of exam (1992-1998). They are also reminded of the changes in paper numbers which have occurred during the last six years.
The general format of the exam pages uses three frames (Figure 2), divided as follows:
Figure 1. The opening page
Figure 2. A 1997 exam paper
Students can quickly reference the Constants or Formulae pages, which are normally at the front of an exam paper, by using the bottom frame links which are always present on the screen. We use extra windows for the constants, formulae and the correct answer, so that the variables (time and score) are not destroyed. Figure 2 shows the screen of the page for the 1997 Introduction to Physics paper (under a different paper number, as they were changed from 1998). Note the use of 'buttons' for the multiple-choice selection, rather than the letters (rendered as hyperlinks) A, B, C, D, and E. The reason is that if the latter are used as HyperText links, the colour changes when clicked, and if the wrong answer is selected, all four wrong answers have the colour changed - an immediate hint to the student that the one remaining unchanged letter is the correct answer!
Figure 3. Table of Constants
Figure 4. Formulae given at the front of the exam paper
Using the exam pages
The student can select an exam to 'sit' from the Exam Index page. As soon as the selected exam page loads the 'clock' at the bottom of the screen starts incrementing, which allows the students to time themselves - as if they were sitting a 'real' exam. The student answers each question by clicking on the appropriate button (A to E). If they select a wrong answer a window comes up which simply tells them that - see Figure 5. When they select the correct answer another window opens to tell them that, and also to show how the correct answer is obtained (see Figure 6).
A score is kept in the bottom frame. The Formulae and Constants windows can be opened at any time, and kept open on the student's computer for easy reference.
Figure 5. Incorrect answer window
Figure 6. Correct answer window
Examples of questions that make specific use of computer screen capabilities
Use of diagrams in both question and answer
Questions that include diagrams are particularly useful for teaching purposes, since the diagram (or at least the correct diagram) can be reused in the correct answer tutorial. An example of diagrams in the question is provided in the following electrostatics example (first frame is the question, second is the correct answer):
Use of colour
One of the important differences between the normal exam format and the computer generated one is the use of colour. This is demonstrated in the following example on colour mixing.
Note - the ability to use colour is a bonus - something that we can't do in an ordinary exam paper, but in the example below we do have to be careful not to give away the answer. For this reason, in the 'question' figure, the proper colours are deliberately not shown in parts of the white area.
Pitfalls to be avoided - and some tips
Student assessment and server statistics
At the end of the PHSI 141 lectures in June 1997, students were asked to assess the exam web pages via an eight-question survey form; 320 students (from a class of 525) completed the survey. The results are summarized in Figure 7.
Figure 7. Some results from a student survey, June 1997
The positive response to Q2 shows that we had done adequate 'advertising' of the exams, but we were rather disappointed by the overwhelming 90% who preferred a hardcopy form of the exam, and the 63% who had not actually used them. We believe that the Q4 result came about because of the difficulties in gaining access to the CAL laboratories - especially for first year students, and we think that this problem has now been resolved. (Note that the weekly requests just before the mid-term exam in 1997 is approximately 10500 compared to approximately 33000 before the 1998 mid-term exam, where the population has doubled).
The equivalent PHSI 110 lectures don't finish until early October 1998 (because the new paper structure has the equivalent paper to PHSI 141 lasting the full academic year), so we will be running a second survey then.
The software package Getstats (v1.2) was used to analyze the server log for all accesses to the physics exams for the period March 1997 to August 1998 (526 days). Figure 8 shows the accesses to all the exam pages on a weekly basis over that time.
Figure 8. Weekly web requests for all exam papers: March 1997 to August 1998
The obvious peaks occur in this distribution just before exams (the real ones!), on 28 April 1997 (mid-term), 11 June 1997 (final), 25 June 1998 (final) and 27 July 1998 (mid-term). It is encouraging to see an increase in the use of the exams over the 18-month period.
The final figure (Figure 9) shows the distribution of paper requests by the year of the paper. As might be expected it is the most recent years which are requested most. We also have the capability of analysing the web statistics in detail - in fact at the individual question level - and this will be part of the thrust of future work.
Figure 9. Requests for all exams as a function of year
This paper has described a new way of providing students with help with their physics learning, using the method of multi-choice exams from previous years, with model answers. The students have a chance at trying the problems themselves, and if they cannot see how the correct answer is obtained they have the advantage (over the hardcopy form of the exam paper) of accessing help in the form of model answers to the problems. By using the Internet as the medium, the exams and answers are available from anywhere at anytime.
The exams have been available to students for eighteen months, and more students are using the system as they become confident in using the CAL facilities on campus. The students will be surveyed again near the end of 1998.
Site Address: http://www.physics.otago.ac.nz/physics_exams/
John L Bähr and Bernard T Darnton
CAL-laborate Volume 2 October 1998
Page Maintained By: PhySciCH@mail.usyd.edu.au