Tutoring on the WWWNathan Scott and Brian Stone
University of Western Australia
This article is reprinted, with permission, from the Teaching and Learning Technology Programme Newsletter, No. 10, Autumn, 1997.
In 1995 at the University of Western Australia we introduced an "intelligent computer tutoring system" for the teaching of first year engineering dynamics. We have now had three years experience with the system and have seen pass rates rise to 95%.
The computer assessment is fairly rigid in that it requires each problem set to be answered correctly before the student may attempt some assessed problems. There are usually about eight problems on a topic designed to find any misunderstandings. Feedback about detected specific misunderstandings is available while these first problems are attempted. Then there are two assessed problems which are marked. These marked problems count for 20% of the unit, the remainder comes from a mid year exam (30%) and a final exam (50%). There are deadlines for the assessed problems after which no marks may be obtained, however the problems must still be completed before the next set of questions may be commenced. In this way all students have to answer all problems correctly, which it is hoped will imply a reasonable degree of competence.
The serving computer records most student actions. Each student is given a printed sheet of the problems during a lecture. This sheet has the same words for each problem that the student will be given by the computer but the numbers are not those the student is given when logged in. Each student has a different set of numerical values for the problem; the server generates these numbers randomly within defined limits. The questions have been designed to "trap" common misconceptions which have been determined by a thorough analysis of past exam scripts. For each question, the software generates the correct answer and several possible erroneous answers that would be expected if the anticipated misconceptions were applied. Any student answer within ±2% of any of the answers is assumed to have been derived by the associated method. Then, as necessary, students are given appropriate feed-back. In dynamics around 200 questions are set over the course.
At any time a student may also send a query on-line to staff. All such queries and staff responses are attached to the problem and may be viewed by all other students. Students may also send personal and confidential queries by a private forum.
Finally, the progress of all students is monitored and displayed so that the lecturer always knows the state of the class:- are they all stuck on a particular problem, is anyone not keeping up, which problems have required the most attempts etc.
The outcome has been an increase in pass rate from around 80% to 95% with exams that have been moderated to ensure that they have not got easier. The same approach has been taken up within UWA for statics and thermodynamics.
Figure 1. First problem of first set
Figure 2. Error message if units omitted
Figure 3. Help card for units
Figure 4. Error message if wrong numbers used
More information about the system can be found at:
Within Mathematics, the approach has been extended to calculus and statistics, with equations required as answers and with diagnostic feedback available.
CalMaeth: Fast, friendly diagnostics in Statistics and Calculus
Calculus and Statistics are among the most difficult first-year courses, and students need lots of practice and instruction. This places a great burden of assignment marking on staff - an expensive business.
Dr Kevin Judd (Mathematics, UWA) has invented a very successful computer-based interactive mathematics tutorial system which is an enhancement of the one used by Scott and Stone. It is unique: students are given personalised assignments and enter mathematical expressions as answers. The system immediately diagnoses errors in these answers, even if they appear in combinations, and it gives clear English feedback to the student.
This development has led to a completely new kind of Mathematics classroom. Students work singly, or in small groups, at the computer terminals on their questions, the system catches most of the mundane errors, and the tutors are freed to deal with students individually and at a much higher level. So far nearly 1,000 students have used this enhanced system, totalling about 30,000 hours of instruction.
Figure 5. CalMaeth poses a simple calculus question
Figure 6. Diagnostic feedback from CalMaeth about the attempt in figure 5
The students say they like the system because it "drills" them and "makes them do the work". Students also like the way the system gives them immediate feedback and is able to diagnose most common errors in the subject.
The CalMaeth system also:
CalMaeth packages are available for intermediate Calculus and Statistics. You can try them on-line at:
For more information, contact:
UniServe Science News Volume 10 July 1998
Page Maintained By: PhySciCH@mail.usyd.edu.au