Managing Mathematics with CALMATThis article was first published in the CTI Mathematics quarterly newsletter Maths&Stats Volume 9, Number 3, August 1998.
Dexter J Booth
School of Computing and Mathematics, University of Huddersfield, UK
The art of good teaching is the art of being organised; lectures ready, clearly delivered with tutorial and homework support. Homework is given out, handed in, marked and handed back on a weekly basis - at regular times and over regular intervals. Learning is work and the student has to develop the habit of regular application if he or she is to succeed. Knowledge acquisition is not an osmotic process; it requires steady graft and continual reflection. If it were otherwise it would not be worthwhile.
But if the student does not participate, no amount of organisation on behalf of the teacher is going to be of worth. The corner stone of a student's participation rests on their willingness to work alone; a willingness that can be stimulated by regular homework followed by regular feedback. Take that away and pretty soon the wall will be breached - the odd lecture is missed - no sanction. Then the odd tutorial - still no sanction. Then large scale absenteeism And then the sanction - failure - but too late. Admittedly not all students follow this pattern but too many do and here is the point at issue; with a large group of first year students how can the habit of regular effort be instilled? How can the teacher ensure that no-one is 'slipping the net'? Clearly weekly homeworks are out of the question with 100+ students and this is where automated systems come into their own.
This paper describes the operation and consequences of a pilot study to use a computer to deliver tutorial activities and continual assessments to a large group of first year undergraduate engineering students1.
The software is called CALMAT and is accompanied by a sister suite called TASMAT. It was developed by and is available from Dr Jean Cook in the Department of Mathematics at Glasgow Caledonian University.
CALMAT is a software suite that contains expository material, tutorial activities, some measure of assessment and a comprehensive management information system that ensures that the teacher can maintain course control with a close awareness of student effort. The expository material ranges from GCSE mathematics through to A-level mathematics (Scottish University Year 1) and covers all the topics that one would expect to be familiar to a competent junior mathematician.
The CALMAT package opens with a module map as indicated in Figure 1.
Figure 1. Module map
The name of the game is to complete modules one at a time in the order prescribed by the teacher. On the introductory module map, each module is indicated by a number enclosed within a grey circle - the lines indicate the sources of prerequisite knowledge for any given module. Canny thought has ensured this map is personal to the student; as the student progresses through the system the circles change colour. Pink indicates that the module has been started and green indicates that it has been completed. It may sound trite but to some students this does act as a spur and none does it deter. Each module is subdivided into small subsections as indicated in Figure 2.
Figure 2. Typical subsection list
To complete each subsection the student has to answer correctly a number of tutorial type questions in sequence. If the student enters an incorrect answer then the student has to start the sequence again. This greatly concentrates the mind. When a subsection has been completed the subsection label turns green otherwise it turns pink to indicate that it has been started but not completed. Return is possible to any part completed subsection at any future time. When all the subsections have been completed satisfactorily the module marker on the module map turns green and the student is credited with having completed that module. Some modules have specific assessments included and the scores for these are recorded separately.
Coupled to this suite of software is a Management Information System that enables the teacher to keep a close watch on everything that each student does. The system records effort by group, by student and by module (Figure 3).
Figure 3. Completion for a group
Figure 3 displays those modules started (S), completed (C) or not done (blank) by each student for a particular tutorial group of students (the names of the students have been blanked out to comply with the Data Protection Act). The performance of any particular student can also be viewed.
In Figure 4 we can see the work done by one particular student. The numbers down the right-hand side represent the accumulated time that this student has spent on each module. Notice that Module 16 is still only regarded as started but has had a significant time spent on it. Why is it not complete? We can look further into this.
Figure 4. Completion for a student
Here, in Figure 5, we see that this student has a little more effort to expend to complete this module. If it were felt that this student was falling behind or was experiencing some difficulties then it is a simple matter to email the student. I found that widespread use of email was very effective in encouraging and maintaining student participation though there were one or two exceptions to this rule. If a student did not respond to email then it was time to contact the course tutor for further action. Fortunately, this was a rare occurrence but when encountered was received favourably by the course tutor.
Figure 5. Details of a particular module for a particular student
Coupled to CALMAT is a suite called TASMAT which permits assessments to be constructed from a pre-defined bank of questions listed by category. The system is quite simple to use. Firstly, the tutor defines a course - here, in Figure 6, I have defined the course ENG, my engineering students. Figure 7 displays the tutorial groups.
Figure 6. TASMAT screen
Figure 7. TASMAT tutorial groups
Next, a test is created from a bank of questions extant within the system. A typical question is shown in Figure 8. Each test can contain up to ten questions and each question in the test can have up to eight options. Each question contains variable parameters and every time a question is asked the values of these parameters are selected at random. There are two modes for a test. In Tutorial mode the test is formative with Help and Solutions available. By selecting Another Question a similar question can be attempted with a different collection of parameter values. The next question in the test is accessed via the Return to Menu button. The test can be executed in Tutorial mode as often as wished but the scores are not permanently recorded. In Assessment mode the test is summative with the Help, Solutions and Another Question buttons disabled. The test can only be taken once in this mode after which the score is permanently recorded.
Figure 8. A sample question
Furthermore, the order in which questions appear is also random when the test is taken in assessment mode. All this functionality ensures that the chances are small that any two adjacent students are going to be answering the same question at the same time. When the test has been set-up, each member of a tutorial group of the course (up to a maximum of 20) is assigned a unique User Code. A total of 24 groups are possible for each course and when the students log into the system they are automatically assigned to their group within the system. Again, as with CALMAT, a comprehensive management system is available. The student test scores are listed per group - see Figure 9.
Figure 9. Student group scores
The performance of any individual student can be ascertained down to the score for each individual question as is shown in Figure 10.
Figure 10. Student test scores
The first test has blanks for each question. This is because the system failed to record the marks across the network when it was first put to serious use. Despite the frustrations that this caused it was possible to note each student's mark and then, as tutor, enter the marks onto the record. This raises another important point. As far as is possible test the technology because this is where the system is at its most vulnerable and there is nothing more likely to put a student off than trying to use a system that keeps falling over.
The course on which I have been running the pilot installation is a two semester course of service mathematics to 120 first year engineering students from a variety of disciplines. The backgrounds of the arriving students varied widely from A-level, through BTEC to GNVQ with all possible shades of interpretation that such experiences provide. A number of students are 'good' at mathematics, a number are not. Some enjoy the subject whilst others look upon it as a necessary hurdle to cross on their way to a career and may even resent having to study it. The course itself is delivered by two one hour lectures each week followed up by a one hour tutorial. In addition to this the students are expected to spend upwards of three hours per week in 'directed unsupervised activity' (DUA) - a euphemism for homework.
In the past the tutorials have been paper based and very poorly attended. As far as the directed unsupervised activity is concerned, because this could never be monitored it was probably never done. In effect, a significant number of the students were not putting in a consistent effort during the year and leaving their learning of the course material to a cramming session in preparation for the examinations. With some students this approach pays off to the extent that they do not fail the examination. With an increasing proportion of the students, however, this style of learning has disastrous consequences.
The course structure
The 1997-1998 delivery of the course was re-structured to take account of the software. Each lecture was delineated against one of the CALMAT modules and the appropriate course notes duplicated (each module comes with a complete set of course notes). Each tutorial was likewise delineated against one of the CALMAT modules and was taken in a computer laboratory. Certain other modules (13 in all) were designated as starred modules and the students were expected to work through these in their own time. This latter formed the directed unsupervised activity. Every three weeks the students sat a TASMAT test during their appropriate tutorial. This structure is clearly laid out in the Module Handbook that is issued to the students at the beginning of the academic year as is shown in Figure 11:
Lecture Schedule for Semester 1
Figure 11. Course structure
The course is assessed by in-course assessment (40%) and final examination (60%). The in-course assessment is built up by collecting points. The seven TASMAT tests contribute a maximum of 40 points but, of course, no-one is going to achieve all 40 that way. As a spur to the students to undergo their directed unsupervised activity and so develop their own self-assessment each completed 'starred' module will gain them an extra point. These extra points they can then add to their test points to accumulate to a maximum of 40 points. A number of students took this aspect very seriously to finish the year with a 100% contribution from their in-course assessment. Needless to say, these very students gained the highest marks in the final examination.
Operation of the course
Apart from one or two minor technical hitches in the early weeks (they did not seem to be minor at the time!) the students have responded to this form of course delivery with remarkable unanimity. The tutorial attendances, instead of dropping down to a trickle as they did in the past, maintained a steady 80% to 90%. The students were enthusiastic in the main and never seemed at a loss for what to do. I arrived at one tutorial at the start of the second semester, before the first lecture of the semester to find a full tutorial group already beavering away and every student doing something different. As can be seen from the analysed questionnaire that was issued towards the end of the course student response was very favourable:
CFN105 Mathematics 1 Final Questionnaire Week 9 Semester 2
Figure 12. Questionnaire analysis
A detailed quantitative analysis of the results of this year's experience would be of little worth as it would be trying to compare two different cohorts of students operating under two different systems; the assessments last year were paper based and fewer in number and the directed unsupervised activity was not monitored. Instead, suffice it to say that the overall success rate was significantly better than hitherto and convinced me that this system was worthwhile. There are just a few caveats:
I have been convinced from my experiences of running this pilot study that the only effective way a computer aided learning package can be embedded into a mathematics course such as this one is to reverse the scenario and embed the mathematics course into a computer culture. In effect, the course becomes a non-human resource centred activity where the teacher becomes a facilitator and the learner becomes freed of temporal constraints to study patterns. Management-speak this may be, but if the student does participate and does succeed then perhaps the process of learning to learn has begun. All that is required of the learner to succeed is the desire to do so and the willingness to react to the immediate feedback on progress that only an automated system can deliver.
Seated before the computer the learner is challenged to make something happen, to act or pursue, rather than merely react or absorb2.
Furthermore, the course becomes so wrapped up inside the technology that the only way for the student to progress is via the technology which acts as an ever vigilant recorder of the student's efforts; not only can the teacher maintain a continual record of a particular student's efforts but, and equally importantly, the student can as well. However, and this fact cannot be stressed too much, the enveloping of a course of study within a computer ethos can only be made effective if an over-arching management information system is part of the deal.
Supplier contact details:
Dexter J Booth
CAL-laborate Volume 2 October 1998
Page Maintained By: PhySciCH@mail.usyd.edu.au