In the section on effective teaching it was suggested that it is useful to think of your course in terms of what it is that you want students to be able to do at the end. Ideally, this will then be aligned with your assessment, which will test whether or not they are able to do these things.
In the last decade, this issue has been one of the most discussed topics in university-level science and technology education. In some countries it has been noted that students can be very proficient at solving calculus-based pen-and-paper problems, while at the same time being unable to describe in scientific terms what is happening around them in the physical world. This has been linked to the fact that assessment is often focused on calculation rather than understanding. In EPFL courses, sometimes the goal is that students can do math-based problems, sometimes the goal is that they can apply concepts to the world around them, and sometimes the course has both goals. Whatever your goal, try to make sure it is represented in your assessment.
This is one reason why it is useful to write at least some of the learning outcomes for your course in terms of verbs which describe observable behaviour. These verbs can define what it is that you want students to do in an assessment. Remember, from a student’s perspective, it can be very frustrating to develop one set of skills and then discover that what is assessed is actually something different. From a teaching point of view it is also counterproductive, since next year’s students will know not to worry about what you say you want from them, but instead to focus on what the assessment will ask of them.
If you administered two different tests on your course to the same students, would a student get similar results on the two tests? Here are a few simple ideas that can help to ensure that your test measures who has learned in your course rather than who is lucky:
- Tests that are short will often have too few questions to be able to test the breath of material covered. This means ‘getting lucky’ on what questions come up plays a bigger and bigger role. Longer tests allow you to test more material and to have multiple test items for a given concept.
- Ambiguous questions mean students have to guess what the examiner is looking for. Those who guess right get the marks. Again, this tests whether a student is lucky rather than good. When writing the exam therefore it is important to try to weed out ambiguous questions. Having someone else try the questions (a colleague, a doctoral assistant) can help.
- Give students a chance to practice. Even if you feel that your instructions are incredibly clear, some students will best understand what you want from them, when they have had a chance to see it in practice. If you can provide them with a mock exam or a mid-term which uses the same format as the final exam, then they will get a feel for what is required of them. In the case of assessment by projects or reports, you might give them some (unmarked and anonymised) examples of work along with the assessment grid and invite them to try grading them for themselves. Again, this will give them a feel for how the grading criteria will be applied in practice. Remember that the more novel your assessment approach, the more likely it is that students will feel stressed about it. Giving them a ‘trial run’ will help to alleviate unnecessary stress for them (and consequently, for you).
One function of an exam is to allow you differentiate between those who have mastered the material, those who are good enough, those who are not quite good enough, and those who are far from good enough. When putting together your exam, try to choose questions with a range of difficulty levels so that those who are not quite good enough will still be able to get some marks, but will at the same time be able to be differentiated from better students.
This is not to say you should reduce your standards (the standard of an exam is defined by where you set the passing mark (4) and the mark for ‘excellence’ (6), and if you feel some of your questions are reasonably accessible, then you can raise the marks required, as appropriate). It simply means that some of your questions should be more difficult than others.
Remember too that the order of questions can make a difference to students’ attainment. Students who struggle with a first question will often feel increased stress which can impede their decision making. Starting an exam with an accessible question, on the other hand, allows students a chance to settle into their rhythm and to focus on the questions. This can be very important to note in contexts where the order of questions is altered from student to student (such as is recommended in the case of multiple choice questions, for example).
You should try to ensure that your instructions are clear.
It can be a good idea, for example, to let students know how many marks they get for each question or each part of a question. This will help them make decisions as to how to break down their time and prevent them spending a long time struggling over a question that is actually worth only a few marks.
When you get towards the end of the term, it can be hard to remember exactly what emphasis you put on one idea or another during the term. One way to make sure that your exam matches what you have taught is to write some test items at the end of each class. This can become the database from which you draw questions. Another approach is to ask students to make up exam questions and to submit them to you. This will give you a good idea if students see the course in the same way that you do. It is also harder for students to criticise an exam, when they have had a chance to contribute questions (obviously, you should reserve the right to tweak the questions or change certain parameters for the exam).
Students will not solve problems as quickly and as fluently as you do. As a result, there is often a tendency for teachers to underestimate the time required to complete a set of problems. One rule of thumb that is sometimes suggested is to time a doctoral student completing the test and then allow students four times as long. Ideally you should also allow some time for students to check their answers (in the real world we want professionals who will check their own answers, so it is good to leave space for that practice in exams too).
Language is a factor to consider in timing the exam. Students who work through a second language – even fluent students – will usually read more slowly than native speakers and will usually take longer to form answers to questions. So, if there is a tight time limit on your exam, the test may end up becoming a test of language skills rather than a test of their skills and knowledge. This is particularly important for tests where written details are important or that require detailed written answers. If many of your students are being tested through a second language then it is a good idea to allow extra time in the exam.