Free Schools: Policy Exchange report finds no positive effect for nearby schools

Henry Stewart's picture
 20
(Contact for press enquiries: 07870 682442)

Today David Cameron is set to announce an expansion of the free schools programme. According to media reports he will justify this on the basis that “free schools do not just help the performance of their pupils, but pupils in surrounding schools”.

This is a direct quote from Policy Exchange report, “A Rising Tide”, (released today) which analyses the impact of free schools on the existing schools that are close to them. But the data in the report actually shows very mixed results and does not provide the basis for this quote or for Cameron’s overall claim.

Do the primaries closest to primary free schools show a faster increase? No



The Policy Exchange report is clear on this: “Schools closest to Free Schools perform in line with national results at primary.” There is no extra increase for schools close to free schools.

An analysis comparing the closest schools with the national average, based on the increase in KS2 results since the year the free school opened, shows virtually no difference. Indeed the only opening year for which the results were different shows slightly worse performance by those closest to a free school.



 

Do the secondaries closest to secondary free schools show a faster increase? No



The report claims that schools closest to free schools perform “better than … national average at secondary”. However this claim is not supported by the data. If we discount those opening in 2011, where Policy Exchange accepts the sample is too small (just five schools), then there is virtually no difference between the change in results in schools close to free schools and those nationally. For 2012 and 2013 openers, the fall nin GCSE results is exactly the same and for 2014 openers there is just a 1% difference.



The argument for free schools is that they introduce competition that will improve the performance of all schools in the area. This is the basis of Cameron’s argument and is the argument tested in the Policy Exchange document. However the data clearly does not support this. Overall the change in results of schools closest to a free school are remarkably similar to those nationally.

Other factors influence the schools involved. In particular 51% of primary free schools and 29% of secondary free schools in the study are in London, according to NUT figures. Given that London schools have, in recent years, improved at a faster rate than those nationally then schools close to free schools would be expected to benefit from this London effect, but do not appear to.

Note: The data above is taken directly from the Policy Exchange report. Copies of the relevant tables are below in the Appendix.

Analysis by prior school performance: No clear causation



Policy Exchange goes on to analyse the effect on nearby schools, according to whether those schools were previously high performing. For primaries they find that in quartile 4 (the lowest 25% of performers), the closest schools increase at a faster rate than similar schools nationally for three of the four years. However the closest schools do less well than similar ones nationally for the other three quartiles..

If this data is used to argue that the presence of primary schools improve the performance of the least well performing schools, it must also be argued that they reduce the performance of the other 75%, which does not seem an impressive result. This analysis is based on relatively small cohorts of data, just 20 or so schools in each of these year cohorts (out of 16,000 primaries in England). It may be that the presence of a free school is not the key factor. The report itself accepts this:

"It should be obvious – but bears setting out explicitly – that such data cannot demonstrate conclusively that any changes seen are as a response to the new Free School. A school appointing a new Head; a change to Academy status; a glut of teachers leaving; a financial crisis – all of these can affect an individual school for better or worse. It should also be remembered that sample sizes in some of these categories is quite small, and correlation should not be mistaken for causation."

For secondaries the data appears to show that less well performing schools close to free schools improve more than similar schools nationally, this time for quartiles 3 and 4. However again the reverse is true for the other 50% of schools. Again the cohorts are small, 20 or so schools in each cohort. And again the schools may have been affected by other factors, such as the London effect.

Do free schools help nearby undersubscribed schools? No



The Policy Exchange report states in its summary: “Primary schools with surplus places show a bigger increase in results than schools which are oversubscribed in every year apart from those approved in 2013.”

Read that sentence carefully. It is not claiming that surplus place schools improve at a faster rate if they are close to free schools. It is simply stating that surplus place schools that are close to free schools improve more than other schools that are close to free schools. It says nothing about any benefit due to free schools.

Indeed if we compare both types (surplus places and not) to national figures, we find neither type of school outperforms:



The data is clear: Schools with surplus places, that are close to free schools, increase at a similar rate to national levels. There is no apparent benefit from competition from a free school.



For schools without surplus places (termed “basic need” schools in the report), they performed better than schools nationally for the 2014 cohort but worse for the 2011, 2012 and 2013 cohorts – and significantly worse for the 2011 and 2012 schools..

The question of why schools with surplus places may improve at a faster rate is an interesting one. It could be due to the pressure to attract pupils, or due to the benefits of smaller classes. However there is no evidence that the presence of a free school leads to schools with surplus places improving at a faster rate than those nationally.

The coalition has been criticised for opening free schools where there are already surplus places. However this has been more on the basis of the waste of money involved than that it will reduce results in nearby schools. This analysis reveals that the introduction of free schools in areas with surplus places brings no extra benefit to those schools and so still seems to be a waste of valuable resources.

Note: This analysis is purely for primaries. Policy Exchange themselves accept the data for secondaries is “inconsistent”.

“At least they don’t do any harm”



This was the question put to me in a Sky News interview, based on the report’s claim that free schools “do not drag down results of neighbouring schools”. My response was that it comes to something when the main claim for a major government policy is that “at least it hasn’t done any harm”.

In fact the data in the report does not support this claim either. The data indicates that, for primaries, schools close to free schools, that have surplus places, perform in line with national levels. However primary schools close to free schools, that are fully subscribed, saw – in three of the four years – their results improve at a slower level than national. If, therefore, the causation is as described in the report then free schools have done harm to nearby fully subscribed schools.

To be honest, I’m not sure that is the case. I think free schools have probably had little effect on neighbouring schools. Any results above or below the national average, in schools close to free schools, are likely to be down to other factors.

So have they caused harm? Well the free schools that have been closed, those that have had their sponsors removed due to poor performance and where the head has been removed due to financial mismanagement – those have all caused harm and indicate a need for greater oversight.

But the greatest harm is in the use of resources that could have been used elsewhere. There has been a £1.7 billion capital spend on free schools and, according to the National Audit Office, 52% of them have opened in areas with little or no need for new places. Cameron will claim they have benefit local under-subscribed schools but the data is clear – they have not, as those nearby schools with surplus places have seen their results increase in line with national figures.

Do free schools and academies have better Ofsted verdicts? No



The government has also made claims that free schools and academies had far better Ofsted verdicts. The Policy Exchange addresses this for free schools but provides no basis to Cameron’s claim:

“Comparing Ofsted grades of Free Schools to all schools over a similar time period does not produce a reliable judgement of quality either for the benefits or weaknesses of free schools.” (P15)

For academies it is also hard to judge. Converter academies have Ofsted verdicts well above the average, but this is understandable given that the schools that converted were primarily already Good or Outstanding. Sponsored academies have Ofsted verdicts well below the average, but again this can be explained by the fact that it was mainly weaker schools that became sponsored academies.

Summary: The creation of free schools does not "help pupils in surrounding schools"



The data in the report is clear. For both primaries and secondaries, there is no benefit to nearby schools from the creation of a free school. Free schools are still an experiment. The first exam results will not be available until 2016. There is no evidence, in this report or elsewhere, of educational benefit from the considerable resources devoted to them.

Appendix: Policy Exchange Data



All of the data in this post is taken directly from Policy Exchange’s report, “A Rising Tide”, and specifically from tables 2,1, 2,2, 2.3, 2.5, 2.6, 2.7 and 2.8:













2

Comments

Janet Downs's picture
Mon, 09/03/2015 - 10:09

Policy Exchange's research is flawed (see here). It should be remembered that Policy Exchange is not independent though it claims to be. It was set up in 2002 by, among others, Michael Gove and Nick Boles (now business and schools minister). It was led for five years by Neil O’Brien is now a Special Adviser to George Osborne at the Treasury. When he was head of PX, the Telegraph wrote:

'Policy Exchange remains to be one of the most influential think tanks in Westminster and is often quoted as Cameron's favourite think tank. Neil O'Brien has succeeded in moving Policy Exchange on to a more aggressive, free-market agenda. Under his leadership the think tank has produced many radical papers on planning reform and education which have gone on to shape government policy.'

As such, any 'independent' research by PX is likely to be biased in favour of this 'aggressive, free market agenda'.




Janet Downs's picture
Mon, 09/03/2015 - 10:20

It should also be remembered that Policy Exchange co-authored 'Blocking the Best', published before the last election, which supported for-profit schools. All that was necessary, it said, was for schools to become 'independent' and they could outsource their running to a for-profit provider (as has happened with the free school IES Breckland).

Academies are 'independent' schools.

When 'Blocking the Best' was launched, Michael Gove, then shadow education secretary, welcomed it and said he would be happy to see groups like Serco running schools.

'Blocking the Best' was co-authored with the New Schools Network, the taxpayer-funded charity which promotes free schools. Policy Exchange's 'Rising Tide' report acknowledges the help given by NSN in the report's production. This raises questions about NSN's independence if it is explicitly feeding into reports which shore up Gov't policy. NSN has already had to be twice reminded that, as a charity, it should remain impartial.








Janet Downs's picture
Mon, 09/03/2015 - 13:23

'Now we have new evidence about Free Schools. And it shows that they’re working.'

Jonathan Simons on ConservativeHome.

The first comment disputes this and it links to Henry's article.


John Wadsworth's picture
Mon, 09/03/2015 - 14:22

It is maybe significant in view of the subject of this report that Jonathan Simons (one of the authors of the report is founder (and chair of governors) of the Greenwich Free School so has a vested interest in putting a positive gloss on the impact of Free schools. For the record, the co founder is Tom Shinner (Director of Strategy at the DfE). GFS is included in the number of free schools that have been judged by Ofsted to "require improvement see here https://news.tes.co.uk/b/news/2014/04/24/flagship-free-school-not-good-e...


Janet Downs's picture
Mon, 09/03/2015 - 16:35

David Laws, just speaking on TV (sorry, didn't notice which channel) in response to Cameron's announcement re increasing the number of free schools if the Tories win the election asked where the money was coming from. Capital expenditure had already been announced, he said, so the money would have to come from money allocated to the repair, refurbishments and rebuilding of existing schools.


Janet Downs's picture
Mon, 09/03/2015 - 16:42

ITV reports Cameron saying free schools have been 'remarkable'. This is a repeat of statements he's made in the past. The New Schools Network said the same thing in October when it reported remarks from Cameron and Nicky Morgan who contradicted each other. Cameron said 'three-quarters' were rated Good or Outstanding while Morgan said it was 'two-thirds'.

Statistics aren't her strong point.




… post from Henry Stewart of the Local Schools Network does the same thing — using Policy Exchange’s own data to refute their findings. …


… Free Schools: Policy Exchange report finds no positive effect for nearby schools | Local Schools Net…. – Henry Stewart …


Janet Downs's picture
Tue, 10/03/2015 - 07:58

I've just looked again at the YouTube clip of the launch of the Policy Exchange document 'Blocking the Best' before the last election. Gove was the main speaker and welcomed the report. As I said above, he told the audience he would be happy to see groups like Serco running school.

Right at the end 58.19 mins there's a question re whether schools would be run for profit in the future. The Policy Exchange rep (next to Rachel Wolf of the New Schools Network with co-authored the report)) said there's a lot of 'emotion' round for-profit school. She said Blair thought it was OK but it was too politically sensitive to pursue at the time. However Policy Exchange would 'nudge' in that direction.

Proof, if any were needed, that Policy Exchange together with the New Schools Network (a charity which is supposed under charity rules to remain impartial) are pushing an agenda which was welcomed by Michael Gove, still an influential Tory despite being demoted to the Whip's office.


Janet Downs's picture
Tue, 10/03/2015 - 08:14

Seems the Policy Exchange report is contradicted by DfE findings. The DfE report into Free Schools and Innovation (Figure 14) said just 32% of free school heads thought their school had 'provided competition' which raised standards in the wider area

The sample size, 53, is, of course, too small to come to any reliable conclusion. But small sample size has not worried the DfE or ministers nor, it seems, Policy Exchange.


… You can read Henry Stewart’s report on the LSN website by clicking on the following link: http://www.localschoolsnetwork.org.uk/2015/03/free-schools-policy-exchan... and the Policy Exchange report can be found here: …


Janet Downs's picture
Tue, 10/03/2015 - 15:15

Schools Week checked Cameron's assertion that the free schools programme has resulted in more new schools being set up since 2010 than at any time in history.

Not so. There were 657 secondary schools built between 1955-1960. The total number of open and approved free schools of all types (primary, secondary, special, sixth-forms etc) is 412. And some of those were already in existence as independent schools.

More dodgy data from the Gov't.



Janet Downs's picture
Tue, 10/03/2015 - 15:48

Speaking on Today yesterday, Jonathan Simons said at least free schools hadn't done any harm. But harm is done if funds are taken from other schools to fund free schools where there's no need for extra places or when money is spent unwisely on potential sites for free schools.

Private Eye, latest edition, gives an example of the EFA spending £1.16m of a 'posh three-bedroomed "historic residence"' for a free school in Reading. The EFA, the Eye says, is only now consulting through Reading council on where to put the proposed school after locals complained the house was in a quiet residential area and wasn't suitable for a 350-pupil primary school.

The consultation names possible sites including some playing fields, a rec and some woodland and meadow. But the EFA has added an extra option - a farm outside Reading in South Oxfordshire.

Janet Downs's picture
Tue, 10/03/2015 - 16:34

A FactCheck by the Conversation found there was no 'empirical basis' for Morgan's statements that free schools were raising standards in nearby schools, were popular or effective. It said:

'It is neither definitely untrue nor definitely true. There are reasons for scepticism, but there is simply insufficient evidence to establish any of the three claims for wider local spill-overs, for popularity nor for effectiveness.'


Janet Downs's picture
Tue, 10/03/2015 - 16:43

Politics UK also finds the claims made in the Policy Exchange report to be invalid. It quotes this passage from the report admitting there's no conclusive evidence.

"It should be obvious – but bears setting out explicitly – that such data cannot demonstrate conclusively that any changes seen are as a response to the new free school."

This, Politics UK says 'fundamentally undermines' the report's basis.


Janet Downs's picture
Wed, 11/03/2015 - 12:31

The New Schools Network has responded to the Policy Exchange report. Naturally, it is positive:

'NSN Director Natalie Evans said in response to the report: "It is very positive to see this early vindication that the free schools programme is achieving exactly what it set out to do – raising standards for schools across the board and increasing the number of good school places. It is critical that free schools can continue opening where they are most needed – not just where they provide new places to plug a gap.'

This response would be laughable if it weren't for the fact that NSN is a taxpayer-funded charity. It is, therefore, funded by public money to promote this nonsense.


agov's picture
Wed, 11/03/2015 - 14:33

Don't most charities exist either to make the founder rich very quickly or to tell government what it is paying to hear?


… What is this all about? The claim – in the case of Beeston Fields, and more generally – is that it’s all about improving education for the children. It must therefore be stated clearly and unequivocally, that there is no evidence that a school becoming an academy has any beneficial effect on educational outcomes – and neither does the setting up of ‘free’ schools. Gove, Cameron et al have frequently asserted that they DO but their use of statistics is dubious and has successfully been ‘rubbished’ by Henry Stewart at Local Schools Network. (See http://www.localschoolsnetwork.org.uk/2013/05/the-academies-illusion-wha... and http://www.localschoolsnetwork.org.uk/2015/03/free-schools-policy-exchan...) …


… about the only mention of education, early in the campaign – has been soundly rubbished by Henry Stewart at Local Schools Network. Again, with the uncertainty before the election now over, we may see the flood gates open for …


… attempt to squeeze a positive spin out of a very small, unclear, premature data set.  Read this account or this account and decide for yourself.  And yet,this has become a ‘truth’. Free …


Add new comment

Already a member? Click here to log in before you comment. Or register with us.