[Random Musings] How I write mock exams

Hey there!

This post will be slightly different from what you've been accustomed to for the last month or so, specifically targeted at folks who want to start writing exams. But first off, I have a disclaimer: this is simply my two cents in writing assessments. The real process of exam writing is a lot more complex and involved, and I don’t think it’s reasonable to expect one or a couple people following the same process for a mock exam – it’s simply overkill. The two parts of “how I write assessments” only describe the process that I follow when writing assessments and that’s IT. If you want to take some bits and pieces here and integrate them in your own practice, that’s lovely, but I’m not saying that my method is the best on Earth.

As you already know, I love writing assessments. It isn't news to many readers here – besides that I’m very happy to see y’all using my tests to prepare for your internals, I enjoy the process of writing tests. They don’t always have to be formal, i.e., mocks, and they may follow a totally different grading scale (i.e., unit tests and the A/E system). The way I write each type of test is slightly different, as the fundamental purpose of the test is different (unit tests are more formative and mocks are more summative), but the core is 85% the same.

Let me get this straight: writing good assessments is a skill, and like any other skill, you’ll need a fair amount of practice before you get decent at it. The first tests I wrote were utter garbage, but I still keep em in a folder and occasionally look back to see how far I’ve grown over the years.

This is the first part of the two-part series on writing summative external assessments, focusing on how to write good questions for an exam. The second part will focus mainly on how to put the questions together to create a coherent exam paper.

Without any further ado, let’s get started.

Step 0: Do a shitload of exam papers

This is crucial, full stop. You want to know how the IB (or other exam boards) assesses the topic you’re writing at different levels of demand. If there’s time, do some rough analysis of the questions you’ve just done and the difficulty of the specific questions – this will help you develop a bank of resources that are similar to what the kiddos will see on their exams. This doesn’t mean you should always keep the same question structures, change the numbers and call it a mock paper – this may get you into trouble for copyright reasons, and students will fall into the familiarity trap – when things don’t go according to plan (say, a topic is being phrased in a slightly different way), kids start panicking because they aren’t exposed to different types of questions.

However, no one is stopping you from looking at stuff from other exam boards. This point is bolded intentionally because I’ve seen so many students (and teachers as well!) refusing to use stuff from other exam boards because they don’t necessarily match with the course you’re teaching. Well, as a writer, you’re expected to read the fine prints in the syllabus, but you can always adapt an existing question template from other places to fit with the syllabus you’re writing the assessments for. Most of them assess the same content, and I can’t see the reason not to look/use questions from other sources to expose yourself as a test writer to a larger variety of questions.

Step 1: Read the examiners’ reports for the questions you do in Step 0

Whenever the examiners’ reports (ER) are available, read them. They always include the common errors that students make, occasionally some data on how students performed on the question, as well as some takeaways for future sessions. If you don’t know where to start, A-level exam boards usually release the ERs when the papers are released. For a great source of examiners reports, look up CAIE examiners reports online. In major exam series, they release 3 variants per season, and that comes with 3 independent examiners reports.

Note that as exam writers, we want some questions to be discriminating. “Discriminating” in exam writing means the question produces a WIDE range of marks for all students and effectively distinguishes the grade 7s from the grade 3s, in IB money. This is a good thing, because we want to ensure that the grade the students receive can be easily defended and attached to their attainment in the subject. (If you’re interested, check out Cresswell’s research papers, accessible at https://www.aqa.org.uk/about-us/our-research. It’s definitely not a bedtime read, but quite an interesting one.)

Steps 0 and 1 should be repeated as much as possible. The more questions you do, the better your second nature is when you read a fresh paper. You could already anticipate where students would find difficult or where students should flourish. A rule of thumb when doing questions, and this only occurs after you’ve done a fair amount: if you have to think about how to do the question, the question is pretty difficult for most students.

Step 2: Understand the syllabus inside out

I’ve briefly mentioned this in step 0 – you must read the fine prints to know what’s expected of students and what is out of the scope of the syllabus. For a class test, this definitely doesn’t matter as much, as you (should) have higher expectations than the bare minimum for your students. However, when writing mocks, we don’t want to cause additional stress and confusion for students with questions that aren’t part of the syllabus. This reminds me of the first time I wrote a mock IB exam, back in 2020, when I anticipated the candidates to rewrite both the numerator and denominator using compound angle formulae just to realise that it’s NOT part of the syllabus and everyone just abused their GDC to find the answer.

          If you don’t have much experience with the syllabus, I strongly recommend having a copy (physical if you’re old school like me) at your disposal. Read the specification and the corresponding examples (this is mostly for maths, but may apply to other subjects) to see what could be asked. This will also avoid missing entire topic(s) from the exam.

Step 3: Analysing the paper as a whole

After doing the real exams, it’s a good idea to briefly analyse the paper. Here are the key details that I focus on:

- how difficult is the question and who is this question targeted at?

- how are the topics spread out in the paper?

- how quickly/slowly is the ramping of difficulty of the paper? Would a weaker candidate have a chance to show off their understanding at a later point?

- (for IB AA section B/AI Paper 2) how is the difficulty increasing throughout the question? Do students at different levels have questions targeted at their attainment? (We’ll ignore the fact that our perception of difficulty is different from the candidates’ – end of the day, we’re just trying to give them a rough structure to follow. It’s not perfect but it’s better than nothing!)

Step 4: Setup a rough table of topics for the paper

Depending on the type of assessment you’re writing, the table will be organised in a different way. In this particular instance I’m talking about writing a mock exam, aka a practice exam following a similar style as the students would see in May/June.

          To start off, I always have a table with two columns (one for each paper) and two rows (one for each section). The content for both papers is determined at the same time, as you don’t want to give too much emphasis on one topic on any paper. This will also ensure that you have a wide range of difficulty for questions in any topic across the two papers so we don’t lose anyone. We’ll go through how to “gauge” the difficulty of the paper in the next post.

Quick example: if complex numbers is an early section A question on Paper 1, it should be a late section A question on Paper 2 or a harder part in a section B question on either paper.

For now, just start anywhere you like (early/mid/late section A), fill in a topic, then move on to another topic until you’ve sufficiently covered the topics in the syllabus. Looking at the syllabus, you would be able to roughly calculate the number of marks devoted to each topic (roughly because some topics are evaluated implicitly as part of a different topic, say, quadratic equations).

Choosing the topics for section B is harder, as you’ll need to link several topics together to develop the questions. We all know the common links – differentiation and integration; trig and integration; trig, complex numbers and integration; vectors and algebra, etc., but it’s hard to write a more unpredictable question. I’m still working on how I could make section B questions slightly more unpredictable.

	Paper 1	Paper 2
Section A
Section B

          A quick disclaimer: you don’t have to test absolutely every topic on the syllabus every single time. If memory serves me, an exam series usually samples about 70% of the content (including the core content) and each topic is assessed at least once every three series.

On predictability

Doing what I just described increases the unpredictability of the exam – if you know precisely what’ll be on the exam, it becomes a checklist exercise: you get marks for every topic you do correctly. In the long term, teachers would focus on superficial teaching (because that’s what you need to do to get the marks) instead of going in depth, and this will only hurt students in the long run. I’m aware that we’re only talking about mocks and not systematic exams here, but we need to take that into consideration to somewhat improve our reliability.

          That being said, a certain level of predictability is expected. Students should know what to expect but they should also know to expect the unexpected, to a lesser content. The majority of questions should be predictable and similar to what they’ve been doing throughout the course. A sizeable minority of questions must be unpredictable to distinguish ones who barely know the content and can only do routine question (standard for a grade 4 student, essentially) and ones who could tackle unfamiliar/non-routine scenarios (standard for a grade 7). This also implies that the questions you used for class tests should be somewhat different from the class exercises (sure, we could keep the majority of the questions, but there has to be some level of development to stretch the highest attainers).

I’ll continue the discussion re writing questions in the next post.

For now, if you managed to make it to this line, thanks for reading and I hope you’ve had some food for thoughts.

Andrew’s out!