| Keynote Speaker | Professor Michael Prosser, Director, Academic Development Unit, La Trobe University
Title: Evaluating the New Technologies Abstract: The impact of the new technologies on the processes and outcomes of teaching and learning in universities is substantial and growing rapidly. The developments are occurring faster than they can be properly evaluated. Much of the evaluation that is being conducted is from the teacher's perspective, focussing on learning gains by students on test produced by teachers, and improvements in the productivity of teaching and learning (Alexander and McKenzie, 1998). There has been little research or evaluation focussing on the students' experiences of using the technology - a student rather than teacher perspective. How do the students experience the new technologies, what do they think are the aims, what do they believe they are learning, how do they approach the use of such technologies? This is surprising, given the impact of the research and evaluation in teaching and learning in higher education from a student learning perspective. It is from this perspective that the major Australian teaching and learning benchmarking instrument - the Course Experience Questionnaire - was developed. This paper will address the issue of evaluation of the new technologies from a student learning perspective. In doing so, it will outline the characteristics of the perspective, show examples of research and evaluation from that perspective, and outline some strategies for the future evaluation of the new technologies in teaching and learning. |
| Keynote Speaker PowerPoint Presentation |
Professor Ann Sefton, Associate Dean (Curriculum Development) Faculty of Medicine, Associate Dean (Curriculum Coordination) Faculty of Dentistry, The University of Sydney
Title: Evaluating the New Technologies Abstract: There is an accelerating trend towards the use of new technologies in teaching; the challenge is to demonstrate its effectiveness. In approaching the evaluation of new methods, what were the aims of the initiators? While their over-riding expectation or hope is usually to enhance learning in a specific area, other possible outcomes may include reaching more students, providing experiences otherwise impossible, offering flexible access to a wider range of information, encouraging rehearsal and practice in virtual environments, the development of more generic skills - including the use of computers themselves. Unless these expectations are made explicit, the impact or effectiveness of the technological solution cannot be measured against its own goals. On-going evaluation in use can subsequently feed into quality improvement cycles. In other situations, comparative judgements are sought but in many ways this approach is difficult. There is by no means agreement on the best methods of evaluation, even for the most basic of questions: Is the technology more effective in enhancing students' learning than are the alternatives it replaces? Is it cost-effective? Is it received better by the students? Conventional teaching methods have by no means always been evaluated rigorously, so the baselines for comparison are lacking or flawed. Complex variables (characteristics of the program in which the technology is embedded, students, teachers) inevitably confound any differences found, so absolute judgements are rarely possible. Nevertheless, some strategies for evaluation have evolved and are often very effective within a local context. Examples from fixed media and web-based technology will be discussed. |
| Paper PowerPoint Presentation |
Geoffrey T. Crisp, University of Adelaide
Title: Is it possible to design a relevant syllabus for level I chemistry? Abstract: Our Department has been involved in a major review of our Level I Chemistry subject over the past 3-5 years. The reasons for the review included a realisation that both staff and students needed to be aware of new paradigms in learning and teaching and that changes in pedagogy would necessitate changes in the presentation format of subjects as well as their method of assessment. Rapid changes in computer technologies are causing staff and students to reconsider the format of their learning and teaching environment. We must regularly assess the importance of current concepts and appropriate modes of delivery for educationally relevant material. In addition to the factual information that students must assimilate, chemical educators need to provide students with a framework within which the information can be used in a constructive manner. Staff need to encourage learning strategies that will be of benefit to lifelong education. Society expects graduates who are critical thinkers and not simply laboratory machines. New discoveries and significant advances in science do not spring from repetition. Research is founded on trying something new, doing the unusual. This is how the teaching of chemistry can keep pace with research, by being innovative and imaginative rather than repetitive. Surveys of students have consistently indicated their preference for a contextual framework for the subject content. One issue we are concerned with at the present time is how much influence should student feedback have on the content of a subject? Some areas of our subjects are not as "popular" as others. Should this influence the design of the syllabus or should we pay more attention to placing the content in context? The visual aspects of chemistry rather than abstract ideas or historical derivations should be emphasised. What the student does with the information is just as important as the information itself. We have departed from the formalism that emphasises that students cannot understand new or advanced topics before having a thorough understanding of all previous, basic concepts. This formal approach restricts students to an historical perspective to chemical problem solving rather than approaches that are likely to be of benefit in the future. We have conducted surveys of the first year Chemistry I (Science majors) classes over a 3 year period and obtained feedback concerning the use of computer aided instruction and assessment. A review by the Advisory Centre for University Education on the introduction of web-based assessment was performed at the end of 1999. Students had some very positive and some negative comments to make regarding the use of the www for learning and assessment. The analysis of student grades for the past 10 years indicates an increasing proportion of students passing as well as an increasing median mark. What is the significance and consequence of this? The corresponding analysis of advanced chemistry subjects does not always match that at the introductory level. |
| Paper PowerPoint Presentation |
James Dalziel, The University of Sydney
Title: Evaluating teaching materials and educational software for their commercial potential: Issues for academics and teachers to consider Abstract: The commercial potential of newly developed teaching materials and educational software is a frequent undercurrent in discussions of modern education. While commercialisation of teaching materials does not necessarily require computer systems, in practice the vast majority of current interest has been provoked by recent developments in computing, and more particularly, the rise of the Internet. This paper reviews some of the major issues that teachers and academics face when considering the commercialisation of educational materials, and presents a framework to assist in the evaluation of potential commercial products and services for the modern educational world. From an educational developer's perspective (academic or teacher), relevant issues include: the ownership of educational materials (intellectual property), the relationship between educational developers and their employers, the role of copyright and patents, and the positive and negative aspects of collaboration. The importance of business planning in the early stages of evaluating the commercial potential of educational materials is emphasised, as is the importance of partnerships with other appropriate organisations. The entrepreneurial skills and dedication required to successfully develop and market commercially viable educational materials are considered, and a sober assessment of the role of the original educational developer in this process is presented. An evaluation of the current commercial opportunities for educational materials is framed within an understanding of current directions in national and global education. |
| Paper PowerPoint Presentation |
Sharon Fraser and Elizabeth Deane, University of Western Sydney Nepean
Title: Doing it Differently in Science - An Evaluation of the Process Abstract: During 1999 a new teaching and learning methodology was implemented in a 2nd year subject, Immunology, in the School of Science at the University of Western Sydney Nepean. Cooperative learning was the vehicle for learning, an approach significantly different from anything the students had experienced before. Throughout the semester both quantitative and qualitative evaluation was systematically undertaken with all participants: students, academic and technical staff. The evaluation tools incorporated into the programme included short questionnaires, longer surveys, independent observation, staff debriefing sessions and student and staff focus groups. Such tools were designed to determine whether the goals and learning objectives that we had established were being achieved, and/or what modifications could be made to more effectively do so. As a result of such feedback, we made revisions to the subject as it was being delivered and developed recommendations for modifications to the subject structure and its delivery to future cohorts of students. |
| Paper PowerPoint Presentation |
P. D. Jarvis, J. E. Humble, R.D. Watson, I.A. Newman, E. Chelkowska and S. Stack, University of Tasmania
Title: Evaluation of Improved Outcomes in Physics Service Courses Abstract: Pressure exists to find non-technological, as well as computer-based, solutions to problems encountered by first year university students learning physics. Both approaches require evaluation. In 1999, we held a CUTSD grant to investigate the effects of various teaching techniques on students' learning and understanding in first-year service courses for students who would not be proceeding in physics beyond first year. Evaluating the results of the techniques that we tried has proved to be more difficult than we had anticipated. We used a combination of comparisons with previous years, of work-books kept by staff, of student opinion expressed in focus groups, of anecdotal feedback obtained during laboratory and tutorial classes as well as of the information gleanable from the University's standard "Student Evaluation of Teaching" forms. Not surprisingly, we have confirmed that there is no such thing as a "one strategy fits all students". Nevertheless, we have evidence that a modified constructivist approach, clearly basing our teaching on close consideration of students' background experiences, rather than the backgrounds which they would be assumed to have on the basis of their qualifications at entry to the courses, does improve their understanding of the material presented. Importantly, there is also evidence that for at least some students the approach improved their enjoyment of the subject. One hopes that this would lead to realistic appraisal of physics in their subsequent careers. For the teachers, we found that the information gained about student learning proved very useful for our reflective assessment of our teaching techniques and processes. |
| Paper PowerPoint Presentation |
Ian D. Johnston and Rosemary M. Millar, The University of Sydney
Title: Is There a Right Way to Teach Physics? Abstract: One of the most significant advances in Physics Education research over the last decade, has been the production of standardized tests for determining students' understanding of basic concepts. As a result, many groups, particularly in the USA, have used student performance in these tests as evidence that specific teaching strategies will improve students' understanding of the subject. In particular, one group have produced a teaching technique called Interactive Lecture Demonstrations involving highly structured use of computers for collection and analysis of experimental data in the classroom. It is claimed that these will produce reliable and substantial improvement in student understanding in a traditional lecture-based curriculum. This paper reports a trial in which, for two years running, these Interactive Lecture Demonstrations were incorporated into a traditional lecture course on introductory physics at the University of Sydney, taught in parallel with several other courses differing only by timetable. Pre- and post- tests were given to all students to determine whether the target class improved as predicted, and how it performed in relation to the control classes. Comparisons of performance were also made on homework assignments and end-of-semester exams, common across all courses. |
| Paper | Robert Newby, Central Queensland University
Title: Innovative Teaching of the Experimental Sciences in Regional Queensland Abstract: Central Queensland University offers a range of experimental science courses both by distance education and by 'face to face' teaching. The latter is spread over four regional campuses using a range of technology. In recent years there has been a blurring of the boundaries between full-time, part-time, and distance education. More significantly the increasing use of technology has seen a blending of the styles and modes of delivery so that all the traditional terms are in need of redefinition (or perhaps abandonment). The practices behind terms such as 'mixed mode' and 'flexible delivery' are starting to make a real difference for students. Despite the plethora of acronyms; VAL (Video Assisted Learning), ISL (Integrated System-wide Learning), students now have access to a range of technology based resources and we can cater for a range of learning styles (and lifestyles). This paper outlines briefly the innovative technology being used and its influence on teaching techniques. At CQU courses are not evaluated systematically by the University but individual lecturers are encouraged to survey students in their courses. The results from these surveys are allowing us to optimise the flexible delivery models for teaching science in regional areas. The influence of factors such as class size, communication medium, student age, course level and support mechanisms are now reasonably well documented. The limitations of different models as perceived by staff and students are briefly explored. |
| Paper PowerPoint Presentation |
Mary Peat, The University of Sydney
Title: Why am I evaluating this thingummygig? Abstract: Who is this evaluation for; what am I actually going to evaluate; and how best can I do it? Evaluations are costly. Even the simplest form of evaluation takes time away from other activities, both student time in class to carry out the evaluation and teacher's time to analyse it and report on it. The starting point for any evaluation is to identify the stakeholders - students, developers, lecturers, management, and others. Students should be consulted to find out if the product/intervention you are evaluating is perceived by them to have an impact on their learning. Developers need to know how easy the innovation was to implement and put into practice and were there any unforseen difficulties. Was it an effective learning experience? Lecturers want to know about learning gains and efficiency. Was this a better way to present the material? If yes, why? Did it free up time for other activities? Would other materials be of additional benefit? Managers need evidence that the time spent on the development was worth while. Did it lead to a greater efficiency? Did it lead to student learning gains? Does the product justify the cost? Have the exam results improved? Will more students come to this university because of the development? It is important to answer the question "what sort of evaluation should I do?" For many innovations or developments, a complete and exhaustive evaluation may not be possible. This presentation will explore ways in which teaching and learning materials are evaluated and offer some helpful hints on how to accommodate the stakeholders requirements without spending all your time on evaluation. |
Page Maintained By: PhySciCH@mail.usyd.edu.au
Last Update: Monday, 30-Apr-2012 16:32:46 EST
URL: http://science.uniserve.edu.au/workshop/evaluate/abstracts.html