John Bald/language and literacy

Some thoughts for Assistants, particularly in France.

I was English assistant in the Lycée de Nanterre, just outside Paris, from 1970 to 71. I had a very enjoyable year - I must have seen nearly every play in Paris - and the school was happy enough with what I did to give me a very good report. Better than I deserved, if truth were told, so here are some things I got right, and a some I'd do differently.

• From our point of view, we're there to improve our French or other language. We don't do that by hanging out with other English speakers. Dr Christopher Smith gave me the good advice of buying a coffee for colleagues, following which they tend to want to speak to you. However you do it, speak to and with your hosts, allowing them, of course, to practise their English as much as they want.
• Go to as many local productions and concerts as you can, and make it to the big regional centres. Everything you see and do in French helps - even reading the programme.
• Read everything you can in the host language and as little as possible in English. Get the news from the local radio, and don't go past anything you don't understand. As my wonderful French teacher Francis Holmes once advised me, "Pick up a French book..."
• Don't go past things you don't understand. Paper dictionaries are cheap as chips second-hand, and the Académie Française dictionary is on the internet. You might also enjoy Adrienne's Le Gimmick for colloquial French and slang.
• Try to put up an English display, and keep it fresh. Take magazines, ephemera with you for the displays.You might try starting a club, perhaps at lunchtime.
• What goes around comes around. You have 12 hours per week, and they may not be filled - if so, do as much as possible, making friends with teachers and offering to help pupils who are having problems. I could have done a lot more, and would have been happier if I had. Try to see as much as possible of the host school, and ask to visit local primary schools. You'll find French teachers very interested in their work and happy to share their ideas. If you find some teachers very good, spend more time with them and learn. Look at their planning and the programmes of work. Especially do this for the English programme - ask the teachers what children are finding difficult, and find ways of helping them. Look at what teachers do that works, and think about why it does, and about what doesn't work. Look at how they talk to children, and why. Email me if you'd like to discuss anything - you'll find a lot on this weblog about tackling difficulties with English, including spelling, and your pupils will love you for any help you can give them. The French system makes, perhaps, fewer allowances for individual learning patterns than the British does.
• Ask the children what they'd like to be able to say, and teach them how to say it. If they write anything, have them try to hold a whole sentence in their head, and write it without copying. This is very helpful for children with literacy problems. English spelling can be awkward, but is often easier than French, which has roughly four times as many silent letters. Have a look at La Gloire de Mon Père for some interesting thoughts on French spelling.
• Above all, enjoy. This will almost certainly be one of the best years of your life.

Interlude - some political thoughts.

First, modern languages. The elephant in the room is mixed ability teaching, which has been forced on heads of department by managerial headteachers who know little or nothing about languages, and everything about New labour managerial systems. In the mixed ability classes I've seen, as an inspector, observer and consultant, almost all lower-attaining children have been completely at sea, while the higher-attaining children have been held back. I have seen heads of department shrug their shoulders in resignation. This is much worse than the old cliché of "playing to the middle" - it is a disaster, whose effects are now being felt at the top in A level decline, and at the bottom in dropout at 14. In the meantime, our national guidance is left in the hands of CILT, a relic of the Inner London Education Authority, and various spinoffs of New Labour such as LINKS, whose boss has no experience of languages teaching, but much of political diversity. It is time for change, and for a new regime based on brain research rather than Nulab politics. 

Second, the brains tax. This is not an idea from Nick Clegg and Vince Cable, but from Labour warhorse Neil Kinnock, by way of the anti-graduate leftists who run the EU. A person who earns more money pays more tax. Quite rightly. Why graduates should be taxed at a higher rate than film stars and footballers is another question. If this tax is introduced, the best and brightest will find another route, and universities will wither. Bill Gates set the example by dropping out of Harvard.  Coalition brings difficulties, and this is a big one. Removing subsidy is one thing - reducing university education to this single dimension is another. If the brains tax is introduced, there will be consequences that go far beyond economics. We will, no doubt, have more private universities, and the levellers will have to find another route for their bulldozer.

And Finally.... Miliband D., the brains behind most of New Labour's misdeeds, son of a professor and founder of the New Left Review (with Tariq Ali), has launched an interesting video on how to campaign for him, here. Make sure the wine is chilled. Thank people for commi'men's they have made, and go on to his blog, which has the ending do' ne'.  They should have taught him how to speak properly. If he is going to use a Cockney accent, he should do so. Systematically and self-conciously leaving the t off words (par', defea', credi', wha' wan' importan' governmen' websi' but not usually in the middle of words, community, party, where they might also be omitted in Cockney) while speaking Oxonian English sounds artificial and ridiculous, as indeed it is.

PS Miliband Minor in a speech to the Labour Conf on U tube was  less consistent, dropping quite a lot of ts, but pronouncing t in moment and that's.  He sounded a lot more natural as a result. We all take some shortcuts in speech.

No progress in detecting visual problems

Guardian, here, has a disappointing piece on the lack of take up of eye testing in schools. This issue is much more important than it appears. Visual problems interfere with learning to read, and not learning to read interferes with everything else. All the last government's rhetoric about equal opportunities is shown up as a sham by evidence such as this. It tried to make people follow formulae as the basis of improving education, and should instead have been focusing on having teachers think about the children in front of them. Visual screening should be part of every school's provision.

Numicon, Wow!!!

Took the preschool pack from Numicon to a very clever boy who is about to start school. In the space of 20-30mins - I did not have a timer - he had moved from counting squares individually to recognising numbers, matching numerals to shapes, understanding odd and even numbers (the even number on the scheme "has a friend beside it"),  doubling, fitting shapes to the baseboard, first using all tiles, then with the 1 omitted, matching tiles to shapes on the baseboard to make pictures, and beginning to understand addition and subtraction. I've never seen anything like it, and neither had his parents. Rocket propelled progress!

So, if Numicon builds the bridges to addition and subtraction, the remaining task is to teach tables, as the link between these, multiplication and subtraction. An approach that works for this is detailed in earlier maths postings on this weblog.

Meantime we all need to get involved with Numicon.

Research - Numicon gets a good write up in Ann Dowker's paper for the last government. It is also behind the Hackney version of numeracy recovery, which is the only scheme to rival the Numicon results. There are plenty of loopholes in this research, as Dr Dowker acknowledges, but there are loopholes in all research of this kind, and they do not necessarily invalidate it. Dr Dowker's paper certainly gives good grounds for further rigorous examination of all the schemes she describes, but particularly of Numicon.

Numicon - early maths made easy by this brilliant resource

Numicon is a series of plastic shapes that make the transition from counting to dealing with numbers, particuarly addition and subtraction, easy and fun. Grace Fernald (see her wonderful Remedial Techniques in Basic School Subjects, 1943) worked by pinpointing individual strengths and weakesses, moving back to concrete examples in specific areas of difficulty, then moving from this to practising their written form.

The consistency of Numicon shapes, and the way they fit together, makes the concrete stage much more explicit and systematic, with very clear links to learning and understanding numerals and numbers. The shapes are also an introduction to abstraction - they are, simply, plastic shapes of uniform construction, and so lack the interference that might be caused by counting with money or sweets. My only reservation at this stage is in the transition from adding to using tables, a key step for multiplicaton and division, and one on which I can find no clear evidence. Nevertheless, If you have a pre-school child, or if any of your pupils have problems with very basic maths, check it out and buy it - they offer a 30 day return should you not share my enthusiasm.

Numicon has been purchased by Oxford University Press, an excellent decision that will lead to more widespread use. I've bought a set, and am looking forward to using it with my younger pupils. I would welcome correspondence. Numicon got a good mention in Dr Ann Dowker's review of resources for children with difficulties in mathematics, and should also be given credit for its role in the Numeracy Recovery scheme developed by The Learning Trust in Hackney.

PS Numicon's tactile elements give it huge potential for people with visual and hearing impairment.

How A level went wrong...but how to put it right?

Anthony Sheldon has an excellent article in the Telegraph, here, on the way A levels were and the way they are now. I had the breadth of English literature he describes in the lower sixth - though not the excellence in history teaching - and S level work in French from a distinguished teacher, the late Frances Holmes, who had a First and a well deserved academic award from the French government. These things have been swept aside by the disastrous misconception of AS levels, which were supposed to give breadth, but which just led to more exams, and to intolerable pressure on weaker candidates, who had to move from GCSE to A level over four or five subjects rather than three - or just two for people who needed to pick up more GCE O levels in the sixth form.

In the International Baccalaureate, Anthony Sheldon has found a solution that fits his school, and indeed some state schools, such as Maidstone Grammar School. The solution is moving downwards in the age range through the international secondary and primary curricula. Is it the right one? GCSE has been devastated, and national testing for younger pupils wrecked by left-leaning council members and officials at the QCA. Buying someone else's system off the shelf may be the best available answer, but there are snags. The British system was always more flexible and adaptable to individual needs than continental ones. For example, if you are not good at maths in France,  by which they mean learning maths the way the French teach maths, you are a failure. The breadth in the IB would not allow the same depth of historical work that Dr Sheldon found in his S level.

We would have done better to stay with the strengths of our own system, rather than wreck it. Can it be rebuilt?  Clearing out the QCA was a good start, though there are plenty of mini-quangos left, notably in modern languages, which is governed by CILT, a relic of the dissolved Inner London Education Authority,  and maths. Would a move to the IB work for people who need careful guidance and support on entry to the sixth form? We need a lot more work, and the people who do it need to be honest and guided by educational rather than political considerations.

Pink Brain, Blue Brain...more to come

Lise Eliot's Pink Brain Blue Brain (One World) has an interesting collection of evidence on the early development of language. Her evidence suggests that girls begin to talk, mostly, about four weeks before boys, and that their sentences soon become more complex. These small differences, she suggests, grow into the larger ones that we see in school tests and examinations, though they can be countered by effective early education and literacy teaching. This piece in the Guardian has more on the same theme. A key effect might be to remove the concept of "hardwiring" from the debate on intelligence, where it is often misleading (cf Baron Cohen's idea of hardwiring in relation to gender differences, and Chomsky's discredited language acquisition device). We don't have wires in our brains, and it does not help to suggest that we do, even by analogy. Our electrical connections work differently, and are more elaborate than we yet know. We need more precision in the way we discuss these matters.

Autism Test

This report of a new autism test developed by King's College, London, may be the beginning of the end of assessing this condition on the basis of behaviour. We can only  hope that the approach may be extended to Asbergers, which is so frequently misdiagnosed as to be a serious interference with clear thinking about the problems it represents. Guardian report here.

Randomised Controlled Trials - Technical Paper

This weblog aims to communicate with everyone interested in literacy, including researchers. The following paper is technical, but is needed to correct the error of trying to impose a single structure on educational studies, which may not always tell us what we need to know. As always, correspondence is appreciated. This is a slightly revised version to that originally posted. See also notes on randomisation of small trials, and Dr Ben Goldacre's paper on randomised trials.

 

Since 2001, Professors David and Carole Torgerson have argued for randomised controlled trials in educational research, and have lamented the lack of them. Their work continuously asserts the benefits of this approach, for example –

The randomised controlled trial is the best method of assessing causality[i] 

The best method of ascertaining whether an intervention is effective or not is through the use of a randomised trial[ii] 

The only research method that can adequately control for external confounding factors is the randomised controlled trial [iii]

This gold standard methodology should be more widely used as it is an appropriate and robust research technique.[iv]

At times their arguments amount to a reasonable suggestion that two fields of enquiry might learn from each other – for example:

Many aspects of health care research have sufficient similarities to educational research that some of the lessons learned by health care researchers over the past two decades are readily applicable to the re-emerging interest in randomised controlled trials methodology by educational researchers[v]

Elsewhere, their confidence in their “gold standard methodology” comes close to totalitarianism. In Carole Torgerson’s review of evidence on phonics, following the publication of the Clackmannanshire study in 2005[vi], she and her co-authors systematically disqualified every study that did not use this method, irrespective of any other strengths they may have had, including the key issue of long-term follow up to check on whether improvements are lasting. Several of the randomised controlled trials admitted by the authors had very serious flaws, including imprecise methodology, lack of follow-up of initial results, and sample sizes too small to detect potentially significant improvements. [vii] 

The last issue is particularly important in view of Professors Torgerson’s convincing arguments on sample size in research. They say that most studies show a benefit at or below half a standard deviation (roughly 5% more progress than a control group), and that studies should be designed to detect this with 80% certainty.  This requires a minimum sample of 64, with the same number for a control group.[viii] Of the 20 studies included in the DfES survey, only three (including an initial study by Johnstone and Watson) met this criterion for sample size. The rest had samples of under 50, and mostly well under 50, with some as low as 12. These studies used a wide range of methods, some to teach initial reading some to teach pupils described as “learning disabled”; there is no consistency in their methodology, and in some cases it is barely described.  Basing conclusions on grouping flawed studies together, an approach that has had a long and sad history in phonics research, might – and did – produce a result that looks favourable to phonics, but it is not reliable science.

Professors Torgerson recognise potential flaws in  studies using randomised controlled trials, including bias occurring in a sample by chance, technical bias, chiefly from a sample that is too small, subversion bias (from someone who does not believe in whatever they have been asked to do), attrition bias (where people lost from a sample may result in the final sample being skewed), attribution bias (in allocating people to one or the other group, in practice v similar to chance bias), reporting bias, dilution bias (aka seepage of a technique from the group being given it to other pupils), and exclusion bias[ix].  

These problems and more are illustrated in a randomised controlled trial led by Professor Greg Brooks, one of the co-authors of the DfES review, with the participation of Carole Torgerson.[x]The study involved the complete Year 7 of a comprehensive school, 155, and its goal was to bring about an overall improvement in spelling using an unidentified computerised system that based spelling on pupils’ own pronunciation. Pupils were allocated to the additional spelling and control groups at random, and rigorously pre- and post- tested using NFER-Nelson reading and spelling tests. The results showed no improvements in spelling as a result of the exercise, and even a dip in reading scores from the group given the extra spelling, though this was corrected later. So, the answer to the question in the study’s title,    Is an intervention using computer software effective in literacy learning?  is, apparently, no.

But is it? First, the study used one computerised program only, and gives no clear reason for its selection against dozens of others. Any skilled teacher investigating the use of ICT to improve spelling would select software carefully, considering the needs of the pupils and the nature of their spelling difficulties. No such process took place – there is no evidence of children or teachers being consulted about the choice of the software, or any analysis of strengths and weaknesses in their spelling beyond the mean test scores.  Next, it is not clear that the software was used properly. Children had to use it, on laptops, in groups of six, for one hour a day. However, one group had to have five two-hour sessions, and another had one two-hour session and eight hour sessions. Here again, teaching experience should have been taken into account and wasn’t. Would any sensible teacher put a child in front of a computer for two hours a day to improve their spelling? The effects on motivation might be predicted. Did the anonymous program’s manual provide for two hours a day? We should at least have been told if it did so. If it did not – and I have never heard of any that did - the error is serious enough to invalidate this group’s scores. Finally, the researchers retested children’s spelling and reading after just two weeks, with a further test after the control group had been given access to the machines. In other words, the pupils were tested three times in half a term, with no follow up to see whether any changes were permanent – or indeed whether there were any longer term benefits that were not immediately apparent.

Professors Torgerson's claims for randomisation’s ability to eliminate sampling bias range from the categoric randomisation only guarantees comparable groups at pre-test to (in the same paper!) the qualified although randomised controlled trials should, in theory, eliminate selection bias, there are instances where bias can and does occur.[xi]  In this study, the second was true.  The authors say that the randomised controlled trial is the only research method that can adequately control for all the unmeasured variables that may affect student outcomes. Randomisation ensures that all potential confounding factors are distributed without bias across the randomised groups, and controls for temporal and regression to the mean effects.  Nevertheless, the group given the extra spelling had significantly more girls than boys, and a higher starting point that had to be dealt with by additional statistical analysis. It is not clear just how large a sample would be needed to meet the claim that randomisation could control for all variables, but it is clearly larger than the sample needed for statistical validity. How “regression to the mean” is meant to operate over a six week period is not explained, any more than the phenomenon itself can be shown to apply to reading and spelling, where there is evidence of increasing divergence as children move into secondary school. [xii]Finally, the agglomeration of the scores for all pupils in each group into a single figure does not allow us to say whether any group of pupils – higher, average or lower-attaining, or those with special educational needs – gained any more or less benefit from the extra spelling than any other. The effect of randomisation here is to not to improve the quality of evidence generated by the study, but to tell us less, and to make the evidence generated less precise. Overall, there is not a scrap of evidence that randomisation added to the value of this study or compensated for its elementary errors. Its only identifiable effect has been to make a bad study worse.

This study shows that sampling is only one element of research design, that randomisation is one of a range of options, and that it is not necessarily the best. There are examples of successful  educational randomised controlled trials  – one showed the effectiveness of extra funding for schools in poor areas in New York, and another the negative response of pupils to an anti-smoking campaign[xiii]. Full consideration of what made these studies successful is beyond the scope of this paper, though it seems that they were concerned with a single issue that was relatively easy to measure, and used large samples. Professors Torgerson estimate that a study designed to show the likelihood of raising by 1 the number in a class achieving 5+ higher grade GCSEs might run to 8-10,000.[xiv]  The key point is whether a sample will give us the information we are looking for,  and the success of some randomised trials does not constitute grounds for rejecting well-constructed studies that use different sampling techniques that are more suited to their purposes.

The final, crucial weakness in Professors Torgerson’s argument for randomisation as an essential factor in educational research is their application of statistical techniques without sufficient consideration of the context of education. They cite with approval a 1997 longitudinal study showing that socio-economic status is a consistent predictor of educational success[xv].  This is a generally accepted view, but the issue is complex and has consequences for researchers. To make an impact on the problem of low achievement by poorer socio-economic groups, we need to pinpoint why they are achieving less, and we cannot do this simply by agglomerating their results with those of other groups and randomising. This is obvious, but Brooks et al did not do it in the study cited above. Could this be another uncontrolled variable? We cannot tell, and yet randomisation is supposed to distribute such variables across the sample. At one point, Professors Torgerson suggest “stratified randomisation” as a possible solution to the problem[xvi]though they appear to retreat from this in their 2008 book. The key point is that educational performance is not the result of chance, but are the result of a complex interaction between children’s experiences, their personal characteristics, including  the way their brain is structured (eg sensitivity to light, dyslexia[xvii])  and their intellect. To investigate these issues we need precision, not randomisation.

The issue’s political dimensions are summarised in this quotation from the former Secretary of State, David Blunkett -

We welcome studies which combine large-scale, quantitative information, on effect sizes that will enable us to generalise.

Generalisation leads to policy, which is, in this case, the commitment of the Labour and Conservative parties to the use of phonics as the main vehicle for teaching reading and spelling in infant schools, which was reinforced by the statistically significant, long-term gains in the Clackmannanshire study quoted above. The imposition of a new criterion for validating research was very useful to opponents of this approach, as it appeared to knock out the  research that showed the strongest evidence in favour of phonics.  As far as I can find, none of these opponents had organised randomised controlled trials themselves, but this did not limit their enthusiasm. Professors Torgerson and Brooks are not responsible for triggering this response, but no discussion of randomised controlled trials in the present context can ignore it. Perhaps the most important point of all, however, lies in the application of generalisations derived from statistical science to educational research. Statistical generalisations are derived from observed tendencies, and are not laws of nature. They need to be tested against established knowledge in whatever field they are applied.

John Bald, independent consultant.

Author, Using Phonics to Teach Reading and Spelling (Sage, 2007).

---

[i]Torgerson, DJ and Torgerson, CJ  Designing Randomised Trials in Health, Education and the Social Sciences: an introduction. (Palgrave Macmillan, 2008) viii

[ii]Torgerson, DJ and Torgerson, CJ (2003) Avoiding bias in randomised controlled trials in educational research: British Journal of Educational Studies 51: 36-45

[iii]Torgerson, CJ and Torgerson, DJ  (2003) The design and conduct of randomised controlled trials in education: Lessons from health care. Oxford Review of Education, 29: 67-80

[iv]Torgerson, CJ and  Torgerson DJ (2001)  The need for randomised controlled trials in educational research. British Journal of Educational Studies 49,3:316-328.  

[v]Torgerson, CJ and Torgerson, DJ  (2003) The design and conduct of randomised controlled trials in education: Lessons from health care. Oxford Review of Education, 29: 67-80  (p68)

[vi]Johnstone, R and Watson, J (2005) The Effects of Synthetic Phonics Teaching on Reading and Spelling Attainment: A Seven Year Longitudinal Study.  Scottish Executive Central Research Unit.

[vii] Torgerson, C, Brooks G and Hall, J (2006) A Systematic Review of the Research Literature on the Use of Phonics in the Teaching of Reading and Spelling. DFES Research Report RR711 

[viii]Torgerson and Torgerson (2008) 130

[ix]Torgerson, DJ and Torgerson, CJ (2003) Avoiding bias in randomised controlled trials in educational research: British Journal of Educational Studies 51: 36-45

[x]Brooks et al (2006) Is an intervention using computer software effective in literacy learning? A randomised controlled trial  Educational Studies 32: 133-143

[xi]Torgerson, DJ and Torgerson, CJ (2003) Avoiding bias in randomised controlled trials in educational research: British Journal of Educational Studies 51: 36-45

[xii]Eg Chall, J et al, (1990) The Reading Crisis, Why Poor Children Fall Behind, Harvard UP.

[xiii]Crain and York (1976) in Torgerson and Torgerson (2001)

[xiv]Torgerson, CJ and Torgerson, DJ  (2003) The design and conduct of randomised controlled trials in education: Lessons from health care. Oxford Review of Education, 29: 67-80

[xv]Robinson, P. (1997) Literacy, Numeracy and Economic Performance (LSE), in Torgerson, CJ and Torgerson, DJ  (2003) The design and conduct of randomised controlled trials in education: Lessons from health care. Oxford Review of Education, 29: 67-80

[xvi]Torgerson, DJ and Torgerson, CJ (2003) Avoiding bias in randomised controlled trials in educational research: British Journal of Educational Studies 51: 36-45

[xvii]See, for example, Reading Through Colour, (Wilkins, A, 2003)  The Learning Brain (Blakemore, S and Frith, U2005) and Pink Brain Blue Brain (Eliot, L 2010)