Do's & don'ts of using online tests

There are little to no e-learnings without tests. You are almost always testing something, whether it’s knowledge level, degree of involvement, or the dealings of the workplace. Testing gives both the developer and the user grip on the learning process.

Using tests

Tests can generally be divided into two categories: the summative and formative tests. The summative tests can be used to determine one’s level and is tied to some sort of evaluation: a pass mark, an 8, or just an overall satisfactory grade. This can be used for the completion of a training, but also as selection criteria for approval to start a training, or as a condition for a particular part of the training (e.g. something that takes part in a classroom). The formatives tests are of an informal kind. They are used to gain an insight into the learning process, and from this feedback both the teachers and the users can learn more. You can offer formative tests as a warm-up for future interested parties: ‘What do you already know about subject X? Do the online test here and measure your knowledge!’, as an interim moment of measurement, or it can be a handy instrument to study the material.

Types of tests

The classic type of assessment for e-learning is the multiple choice test, which can range from quizzes to official exams and are most effective when testing specific knowledge through question and answer assignments. In order to measure someone’s skills and progress, there are alternative methods available, such as simulations, problem solving in case studies, forums and discussions, group projects, oral tests (video, audio, pitch), assignments for a product (a reflective report, portfolio (photo of) a design, etc), with all of these methods working very well online too.

Characteristics of a good test

Whether the test you use is formative or summative, it needs to be meaningful. Both teacher and user want to feel like they experienced something useful. There are three things you should take into account:

1. Is the test valid? Does it actually measure the learning goals or learning outcomes? Or does it measure something else? This is about the content as well as the type of test. For example, is it more important for a user to be able to apply the knowledge in different situations, then it is probably better to go for open questions rather than multiple choice ones. Miller’s Pyramid (1990) can help to make the best use of your test questions. 

   

2. Is the test reliable? Does it measure consistently? Would a user get a similar result the second time? And what influence does test wiseness have? (More about that soon). Does everyone have a chance of success?

3. Is the test transparent? Is it clear to the user what exactly the test is measuring and how it can be achieved beforehand?

   
   

Developing a test

The aforementioned criteria are useful when developing a test: it needs to be valid, reliable, and transparent. 

There is another phenomenon that needs to be taken into account when developing a multiple choice test, which is test wiseness. Millman (1965) defines test wiseness as the opportunity one has to use the characteristics of a test, or the context in which the test takes place, to their advantage in order to get a higher score. The user could be able to deduct the correct answer through particular patterns in the test format, independent from the content. For example, if the correct answer has been A four times in a row, the fifth time it should probably be B.

A useful tip is to position the answers in alphabetical or chronological order so you will not have any influence on the placements of the correct answers, which limits the risk of users discovering any existing patterns at all. In cases of answers containing sentences, you should ensure there is enough nuance (in all answer options) and that all answers are about the same length, otherwise the observant user will pick out the correct answer immediately. Finally, there is a frequent mistake that the correct answer for question 3 will be given away later, for example in question 7.  

In cases of ‘test your own knowledge,’ it is not completely wrong to take a guess at the answer, because there is also ‘the educated guess.’ Users will still be able to see whether they answered correctly, and maybe even read the feedback as well. And voila, still learned something new! However, you want to avoid these guesses with summative tests that get assessed with a grade, or determine whether someone will get certified or not.

Start growing today!

There are multiple features within Rakoo Academy that can help you develop a great test format. You can choose to add an exam, measurement moment, or assignment to your e-learning, containing all the different types of questions, of which you now know the dos and don’ts!

Download the whitepaper 'Benefits of Online Academy' today and start growing!