Assessment and evaluation in the language classroom

What this heading covers

How a language teacher finds out where a learner stands, and what the paper means by each of the words it uses for doing so. Evaluating proficiency in the four skills has a lesson of its own, and so does remedial teaching — what you do once you have the answer.

The single test that decides most questions in this whole area:

Could a learner score well at this without being good at the skill it names?

If yes, the assessment is measuring something else: memory, handwriting, or how much happened to be revised. Nearly every wrong option in this part of the paper fails that test, and nearly every right one passes it.

Four words that are not synonyms

The paper separates them, so it can ask about them.

Word	What it is
Test	One instrument — a written paper, an oral question, a dictation
Measurement	Assigning a number. "She scored 7 out of 10"
Assessment	Gathering evidence about what a learner can do, by many means
Evaluation	Judgement and decision on that evidence — "she follows what she hears but cannot organise a paragraph, so the next fortnight is writing"

A test is one means of assessment, not the same thing as it. Measurement gives a number; evaluation gives a meaning. And assessment that changes nothing about the next lesson has not finished.

In one line — a test is an instrument, measurement is a number, assessment is the evidence, and evaluation is the decision that follows from it.

Assess the skill, not the memory

A language test should sample the skill it claims to measure.

• Reproducing a memorised answer about a chapter tests recall, not comprehension.
• Reading an unseen passage and inferring from it tests reading.
• Marking a content answer down for spelling tests spelling, whatever the paper says it is testing.
• A dictation mostly tests spelling and handwriting. It is a poor measure of listening, and that is a standard distractor.

Why it goes wrong so reliably

Because the four skills are not equally easy to mark. Writing leaves a page; speaking and listening leave nothing. So assessment drifts towards whatever can be collected and marked at a desk, and the two oral skills — the first two a child develops, and the two most used in life — go unassessed. Once unassessed, they stop being taught, because what is not examined is not prepared.

That chain is the single most examinable idea in this heading.

The kinds of test

Kind	What it is for
Placement	Deciding where a learner should begin
Achievement	What has been learned from a particular course or unit
Proficiency	Overall command of the language, independent of any syllabus
Diagnostic	Locating specific weaknesses, so teaching can address them
Aptitude	Predicting how well someone is likely to learn a language
Formative / summative	During the teaching to shape it; at the end to sum it up

Three distinctions that decide questions:

• Diagnostic against achievement. An achievement test asks how much has been learned; a diagnostic test asks where exactly the difficulty is. A test with many items on one narrow point is diagnostic. An achievement test gives a mark; a diagnostic test gives a location.
• Achievement against proficiency. Achievement is tied to what was taught; proficiency is tied to no syllabus and asks what the learner can do with the language at all.
• Objective against subjective. Objective items (multiple choice, matching, true or false, gap-fill) have one markable answer: reliable, wide in coverage, narrow in what they can reach. Subjective items (essay, oral, project) reach the productive skills but need rubrics to be marked consistently. Only the second kind can assess speaking or extended writing.

Formative and summative

	Formative	Summative
When	During the teaching	At the end of it
Purpose	To improve the learning	To report on it
Form	Observation, questions, classwork, drafts	Term test, board examination
Who uses the result	The teacher, in the next lesson	The record, the certificate

Assessment for learning is formative; assessment of learning is summative; assessment as learning is the learner judging their own work. At the primary stage the intended balance is heavily formative.

In one line — the kind of test is decided by the question it answers: where to begin, how much was learned, what the learner can do, or exactly where the trouble is.

What makes a test a good test

Four qualities, and the paper asks for the distinctions between them.

Quality	What it means	A failure of it
Validity	It measures what it claims to measure	A comprehension paper in tiny print also measures eyesight
Reliability	It gives a consistent result — on a second occasion, or with a second marker	Two teachers marking the same essay ten marks apart
Objectivity	The marker's opinion does not enter the result	An essay marked without criteria
Usability	It can actually be run in the time, cost and effort available	A one-to-one oral for sixty children in one period

The kinds of validity, and of reliability

Both words are plural in practice, and a question may name one kind.

Validity	What it asks
Content	Does the test cover the syllabus it claims to, in proportion?
Construct	Does it actually test the ability it names — is this really listening?
Criterion	Does it agree with another accepted measure — predictive of later performance, or concurrent with an existing test?
Face	Does it look like a fair test of the thing, to the learner and the parent?

Reliability, checked by	How
Test–retest	The same test again after an interval
Parallel forms	Two equivalent versions of the test
Split-half	Comparing the odd-numbered items with the even
Inter-rater	Two markers on the same scripts

The rule that is asked most

• A spelling test is highly reliable and an invalid measure of speaking ability. Consistency is no guarantee of relevance.
• Validity matters more than reliability. Measuring the wrong thing precisely is worse than measuring the right thing roughly.
• A valid test must be reliable; a reliable test need not be valid. Reliability is a necessary condition and nowhere near a sufficient one.

Item analysis

After a test, two numbers tell you whether the questions themselves were any good.

• Difficulty index — the proportion of learners who got the item right. An item everybody answers, or nobody answers, separates nobody; the useful range sits in the middle.
• Discrimination index — whether the item was answered better by those who did well overall than by those who did badly. An item the weaker group answers better is faulty — usually ambiguous, or mis-keyed.

In one line — validity is measuring the right thing and reliability is measuring it consistently, and of the two, measuring the right thing roughly beats measuring the wrong thing precisely.

Criterion and norm referencing

	Norm-referenced	Criterion-referenced
Compared against	The other learners	A stated standard
Answers	"Where does she rank?"	"What can she do?"
Reported as	Rank, percentile, position	"Can write a six-line paragraph with correct tense"
Belongs in	Selection, when places are limited	The classroom

The classroom position is criterion-referenced, and the reason is examinable: a rank tells a child they are behind somebody and nothing about what to learn next. Learning outcomes are written as criteria for exactly this reason.

Discrete-point and integrative testing

	Discrete-point	Integrative
Tests	One item at a time — a tense, a preposition, a word	Several abilities working together, as in real use
Example	Twenty gap-fills on articles	Cloze, dictation with comprehension, an interview, writing a letter
Strength	Precise, easy to mark, diagnostic	Closer to what using a language actually is

The cloze test is worth knowing by name: words are deleted from a passage at regular intervals and the learner supplies them. It works because filling the gaps needs grammar, vocabulary and an understanding of the whole passage at the same time — so it samples overall reading proficiency rather than one isolated feature.

A related pair: direct testing assesses the skill itself — you assess speaking by having someone speak. Indirect testing assesses something believed to underlie it — a written grammar paper offered as evidence of speaking. The syllabus prefers direct testing wherever it can be managed.

What a test does to the teaching

Backwash, also called washback, is the effect an examination has on what is taught before it. It is not a side issue; it is the reason the syllabus cares about test design at all.

• A paper of only memorised answers produces classrooms of memorising, whatever the curriculum says.
• A paper that never assesses listening and speaking produces classrooms that never teach them.
• A paper that sets unseen passages and real writing tasks produces classrooms that read and write.

So the examination is a statement of what a school actually values, and changing the teaching usually means changing the test first. This is the strongest single argument in the heading, and a question asking why a board changed its question paper is asking for it.

In one line — a test does not only measure the teaching, it shapes it, so the quickest way to change what a class does is to change what it is examined on.

Marks, grades and reporting

• Marking reports a number; grading reports a band.
• Absolute grading fixes the bands in advance; relative grading fixes the proportion in each band, so the top few get the top grade however they did — which is norm-referencing by construction.
• Grades were brought into school assessment to blunt a false precision: the difference between 61 and 63 is usually noise, and reporting both as one grade stops a school from treating noise as a ranking.
• A comment the learner must act on is worth more than either, and a mark written beside a comment tends to erase it.

Continuous and comprehensive evaluation

Two words, two separate meanings, and the paper separates them:

• Continuous — through the year, not in one examination.
• Comprehensive — the whole range of what a learner can do: all four skills, and the scholastic and co-scholastic together.

Section 29(2)(h) of the Right to Education Act 2009 requires continuous and comprehensive evaluation of a child's understanding and their ability to apply knowledge. Two provisions from the same Act come up alongside it: no board examination before the end of elementary education, and the no-detention provision, which the 2019 amendment replaced with an examination in classes V and VIII followed by additional instruction and a re-examination.

Assessing only what is easy to mark does not meet the requirement — which brings it straight back to listening and speaking.

The tools

• Observation, the most useful instrument at this stage and the least used.
• Rubrics, checklists and rating scales.
• Portfolios of work over time.
• Anecdotal records — short, dated, factual notes of what a teacher noticed.
• Self-assessment and peer assessment, which teach a learner to look at their own work.
• Projects and group tasks.

Questions of the kind the paper sets

A test gives nearly the same result when the same class takes it again a fortnight later, but it asks only about the footnotes of one chapter. The test is — (a) valid but not reliable (b) reliable but not valid (c) both (d) neither

An achievement test differs from a diagnostic test in that the diagnostic test — (a) is longer (b) locates where exactly the difficulty lies, rather than reporting how much was learned (c) is given at the end of the year (d) carries more marks

"Can read aloud a six-line paragraph with correct phrasing" as a report of achievement is — (a) norm-referenced (b) criterion-referenced (c) relative grading (d) a percentile

A cloze test is described as integrative because — (a) it integrates two languages (b) it draws on grammar, vocabulary and overall understanding at once (c) it is marked by two examiners (d) it combines oral and written work

A board changes its paper so that half the marks come from an unseen passage and a writing task. The effect on classroom teaching is called — (a) validity (b) backwash (c) moderation (d) standardisation

An item that the weaker half of the class answers better than the stronger half most likely has — (a) a high difficulty index (b) a faulty discrimination index — it is ambiguous or mis-keyed (c) good construct validity (d) high reliability

Assessing speaking by means of a written grammar paper is an example of — (a) direct testing (b) indirect testing (c) a diagnostic test (d) criterion referencing

"Comprehensive" in continuous and comprehensive evaluation means — (a) a test every week (b) covering the whole range — all four skills, scholastic and co-scholastic (c) the whole syllabus in one paper (d) evaluation by the whole school

Mistakes that cost marks here

• Treating measurement, assessment and evaluation as the same word.
• Thinking a reliable test must be valid. The implication runs the other way.
• Taking a dictation as a test of listening.
• Setting a reading test on a passage the class has studied, which converts it into a memory test.
• Choosing a norm-referenced report as the classroom answer.
• Confusing diagnostic with achievement — a location against a mark.
• Calling the cloze a discrete-point test. It is the standard example of an integrative one.
• Ignoring backwash, and expecting teaching to change while the examination stays as it was.
• Reading "continuous" as "more frequent tests".

In one line

A test is only as good as the thing it actually samples, so the questions to ask of any assessment are whether it measures the skill it names, whether it would say the same thing twice, what decision it leads to — and what it will make the next term's teaching look like.

In the classroom

Take the last paper you set and mark each question with the skill it really tests, not the skill it is headed with. The gap between the two is the whole of this lesson, and it is usually larger than anyone expects.

Then ask the harder question: if a colleague taught only to that paper, what would the children end up able to do? That is backwash, and the answer is the honest statement of what the paper is for.