CAL-laborate Volume 2 October 1998










*******

Student Multi-Choice Exam Revision Using the Internet

John L Bähr and Bernard T Darnton
Physics Department, University of Otago, NZ

Abstract

The Physics Department of the University of Otago has for many years, assisted students in their exam revision by supplying past exam papers for 100-level (first year) exams. A list of the correct answers to the multiple-choice section of these exams has also been provided, but these only state which of the multiple choices is the correct answer. The students are not shown why it is the correct answer, unless they seek help from a staff member. A number of years ago some slide/audio-tape sets were produced, and more recently, exam questions and answers (for 1987-1990) were put onto HyperText Stacks, but the availability (in place and time) was too severely limited to be of use to the majority of students. As first year student numbers now exceed 1000, a more accessible and efficient method of delivering the exam papers to students was required.

We are now using the Internet, to make past exam questions and answers (with working and explanation) widely accessible to students - from anywhere, at anytime, and from any computer platform. With this method we can also gather useful statistical information relatively easily about the process. This paper describes the setting up of such web pages, and how we provide students with help with their physics revision, using the method of multi-choice exams from previous years together with model answers. Statistical information on question access, and results from a student survey are also presented.

Background and history

Why do exam revision?

The learning of physics is enhanced by solving problems, and one good source of problems is past exam papers, especially multiple-choice exam questions. The Physics Department of the University of Otago has for many years, assisted students in their exam revision by supplying copies of past exam papers for 100-level (first year) exams. A list of the correct answers to the multiple-choice section of these exams has always been provided, but these only state which one of the five multiple choices is the correct answer. In order to enhance their learning from past exam questions, and to relieve the pressure on staff answering the same questions over and over again to queues of students at exam time, we have ported the exam questions to the Internet. Thus students can sit at a computer in one of 15 locations on campus, bring up the appropriate web page using a suitable browser, and 'sit' the multiple-choice section of a past exam paper by clicking on the hypertext-links. This use of the Internet makes past exam questions and answers widely accessible to students - anywhere, anytime, and from any computer platform.

First Year paper structure at Otago

The number and type of physics papers offered to first year students at the University of Otago has changed many times in the last twenty years. The present structure (see Table 1) divides students into 'physical science majors' and 'health science majors', and consists of two papers (PHSI 131 and 132) in each semester for the physical science students and two full year papers (PHSI 110 and PHSI 120) for the health science students. (The latter have lectures at half the rate of the former).


Table 1.


Semester 1 Semester 2
Physical Law & its Applications (PHSI 131)Fundamentals of Modern Physics (PHSI 132)
Introduction to Physics (PHSI 110)Introduction to Physics (PHSI 110)
Physical Universe (PHSI 120)Physical Universe (PHSI 120)


The physical science papers assume an understanding of Bursary (final year of high school) physics and mathematics. The health science papers do not assume an understanding of high school physics, and so the basic mathematics required (non-calculus mathematics) is incorporated in the paper.

The exam for each paper consists of two sections:

  • 30 multiple choice questions; and
  • 4 written-answer questions from a choice of 8.
  • Each section is worth 50%.

    The use of HyperText Markup Language

    The 'de facto' standard for documents on the Internet is HTML. A standard 'web page' is written in this language, which is relatively easy to use, and there are many web page editors available for this. The creation of physics web pages must meet a number of criteria, including the ability to produce superscript/subscript, Greek letters, equations, and diagrams.

    We have also used JavaScript (McComb 1996) for handling the time, score, opening new windows, and enabling the filling of more than one frame at a time.

    Description of the project

    General form of the pages

    Figure 1 shows the opening page for the exams. From here the students select, via the two 'pull-down' menus, the paper and year of exam (1992-1998). They are also reminded of the changes in paper numbers which have occurred during the last six years.

    The general format of the exam pages uses three frames (Figure 2), divided as follows:

  • top frame containing header information;
  • bottom frame containing elapsed time and exam score - for student's information only - and three links to
    • the exam index page
    • a page of physical constants (see Figure 3)
    • a page of useful formulae (see Figure 4) (these latter two are always included in the 'real' exam paper); and
  • centre frame containing the exam paper itself.
  • Figure 1.

    Figure 1. The opening page



    Figure 2.

    Figure 2. A 1997 exam paper



    Students can quickly reference the Constants or Formulae pages, which are normally at the front of an exam paper, by using the bottom frame links which are always present on the screen. We use extra windows for the constants, formulae and the correct answer, so that the variables (time and score) are not destroyed. Figure 2 shows the screen of the page for the 1997 Introduction to Physics paper (under a different paper number, as they were changed from 1998). Note the use of 'buttons' for the multiple-choice selection, rather than the letters (rendered as hyperlinks) A, B, C, D, and E. The reason is that if the latter are used as HyperText links, the colour changes when clicked, and if the wrong answer is selected, all four wrong answers have the colour changed - an immediate hint to the student that the one remaining unchanged letter is the correct answer!

    Figure 3.

    Figure 3. Table of Constants



    Figure 4.

    Figure 4. Formulae given at the front of the exam paper



    Using the exam pages

    The student can select an exam to 'sit' from the Exam Index page. As soon as the selected exam page loads the 'clock' at the bottom of the screen starts incrementing, which allows the students to time themselves - as if they were sitting a 'real' exam. The student answers each question by clicking on the appropriate button (A to E). If they select a wrong answer a window comes up which simply tells them that - see Figure 5. When they select the correct answer another window opens to tell them that, and also to show how the correct answer is obtained (see Figure 6).

    A score is kept in the bottom frame. The Formulae and Constants windows can be opened at any time, and kept open on the student's computer for easy reference.

    Figure 5.

    Figure 5. Incorrect answer window



    Figure 6.

    Figure 6. Correct answer window



    Examples of questions that make specific use of computer screen capabilities

    Use of diagrams in both question and answer

    Questions that include diagrams are particularly useful for teaching purposes, since the diagram (or at least the correct diagram) can be reused in the correct answer tutorial. An example of diagrams in the question is provided in the following electrostatics example (first frame is the question, second is the correct answer):

    Which one of the diagrams below best represents the form of the electric field lines of a pair of positive charges of equal magnitude, placed a small distance apart on the horizontal axis?

    Diagram 1.


    PHSI 141 - 1994, Question 14, Correct

    Diagram 2.

    Field lines always begin on a positive charge and end on a negative charge and
    spread from each other.

    Use of colour

    One of the important differences between the normal exam format and the computer generated one is the use of colour. This is demonstrated in the following example on colour mixing.

    Note - the ability to use colour is a bonus - something that we can't do in an ordinary exam paper, but in the example below we do have to be careful not to give away the answer. For this reason, in the 'question' figure, the proper colours are deliberately not shown in parts of the white area.

    Beams of red, green and blue light of equal strength are directed at a white screen, in a darkened room. When an opaque object is placed solely in the green beam the colour of its shadow on the screen is:

    Diagram 3.

    (A) red (B) yellow (C) magenta (D) cyan (E) black



    PHSI 141 - 1995, Question 7, Correct

    Diagram 4.

    The three colours mix to give white, but with green omitted only the red and blue are left to mix, giving magenta.



    Pitfalls to be avoided - and some tips

    Avoid designing diagrams on a 1024x768 screen and expect them to look good on a 640x480 screen. Trapping code should be used for the Netscape version number, as the time/score doesn't work on Netscape v2. Maths symbols can be unreliable, and this depends on the browser used. For example, the symbol 'sqrt' may need to be used instead of a Ö. The Macintosh version of Netscape v2 has a bug that necessitates a window-opening instruction being repeated. In general, one should design for the intended environment. Our main environment (Macintoshes running Netscape browser) is quite controlled and so we know the fonts needed are installed, and the JavaScript works with every machine it will meet in the environment.

    Student assessment and server statistics

    Student assessment

    At the end of the PHSI 141 lectures in June 1997, students were asked to assess the exam web pages via an eight-question survey form; 320 students (from a class of 525) completed the survey. The results are summarized in Figure 7.

    Figure 7. Some results from a student survey, June 1997



    The positive response to Q2 shows that we had done adequate 'advertising' of the exams, but we were rather disappointed by the overwhelming 90% who preferred a hardcopy form of the exam, and the 63% who had not actually used them. We believe that the Q4 result came about because of the difficulties in gaining access to the CAL laboratories - especially for first year students, and we think that this problem has now been resolved. (Note that the weekly requests just before the mid-term exam in 1997 is approximately 10500 compared to approximately 33000 before the 1998 mid-term exam, where the population has doubled).

    The equivalent PHSI 110 lectures don't finish until early October 1998 (because the new paper structure has the equivalent paper to PHSI 141 lasting the full academic year), so we will be running a second survey then.

    Server statistics

    The software package Getstats (v1.2) was used to analyze the server log for all accesses to the physics exams for the period March 1997 to August 1998 (526 days). Figure 8 shows the accesses to all the exam pages on a weekly basis over that time.

    Figure 8.

    Figure 8. Weekly web requests for all exam papers: March 1997 to August 1998



    The obvious peaks occur in this distribution just before exams (the real ones!), on 28 April 1997 (mid-term), 11 June 1997 (final), 25 June 1998 (final) and 27 July 1998 (mid-term). It is encouraging to see an increase in the use of the exams over the 18-month period.

    The final figure (Figure 9) shows the distribution of paper requests by the year of the paper. As might be expected it is the most recent years which are requested most. We also have the capability of analysing the web statistics in detail - in fact at the individual question level - and this will be part of the thrust of future work.

    Figure 9.

    Figure 9. Requests for all exams as a function of year



    Conclusions

    This paper has described a new way of providing students with help with their physics learning, using the method of multi-choice exams from previous years, with model answers. The students have a chance at trying the problems themselves, and if they cannot see how the correct answer is obtained they have the advantage (over the hardcopy form of the exam paper) of accessing help in the form of model answers to the problems. By using the Internet as the medium, the exams and answers are available from anywhere at anytime.

    The use of JavaScript language means that useful additions to the exam papers, such as a table of physical constants and formulae, can be visible simultaneously, as they would be in a genuine exam paper. Other possibilities in using this medium are the use of colour and animation. The former has been demonstrated in this paper, and while we experimented with some small animations, so far none have been put in place. This may be one of the aims of future work, although this will probably mean departing somewhat from the original 'exam paper' format. In addition, the advantages of animations have to be weighed against the cost in terms of increased creation time and increased download time.

    The exams have been available to students for eighteen months, and more students are using the system as they become confident in using the CAL facilities on campus. The students will be surveyed again near the end of 1998.

    References

    Site Address: http://www.physics.otago.ac.nz/physics_exams/
    McComb, G., (1996) The JavaScript Sourcebook, John Wiley & Sons.

    John L Bähr and Bernard T Darnton
    Physics Department
    University of Otago
    Dunedin
    New Zealand
    bahr@physics.otago.ac.nz


    Return to Contents

    CAL-laborate Volume 2 October 1998

    [an error occurred while processing this directive]

    Page Maintained By: PhySciCH@mail.usyd.edu.au
    Last Update: Monday, 30-Apr-2012 15:14:53 AEST
    URL: http://science.uniserve.edu.au/pubs/callab/vol2/bahr.html