LBWF, Mini-Holland, and air quality: the King’s College Environmental Research Group report and its frailties
Acknowledgement: I am very grateful to Steve Lowe for drawing my attention to the subject of this post, providing important source material, and making helpful suggestions right the way though the drafting process.
In the early summer of 2018, LBWF commissioned the much respected Environmental Research Group based at King’s College London [hereafter KCERG] to ‘model a range of interventions around air quality, exposure and attitudes, and its [sic] impact on the Public in Waltham Forest’.
The background was well known. LBWF had been implementing a £27m. scheme, dubbed ‘Mini-Holland’, which prioritised cyclists and pedestrians over motor vehicles, and thus, it was hoped, secured long-term and significant health improvements. By commissioning KCERG, LBWF aimed to ascertain, for an important indicator, whether and to what extent its goals were being realised.
As far as can be ascertained, KCERG’s methodology was as follows.
It started by estimating levels of air quality across Waltham Forest at two points in time, 2013 and 2020.
The model underpinning these estimates already existed, and was based upon the London Atmospheric Emissions Inventory, much used previously by KCERG. What KCERG did was to modify this model, in particular incorporating ‘the impacts of ULEZ and other London policies’, and, more pertinently, for 2020 adding in ‘extra information specific to Waltham Forest such as the Mini-Holland scheme, interventions aimed at reducing the dominance of motor traffic through traffic calming, strategic road closures in areas such as “town centres” and ‘The Villages”, and further speed caps’.
Armed with the estimates for 2013 and 2020, KCERG next related them to detailed information about Waltham Forest’s population distribution, dividing the borough into 5×5 metre grid squares, and for each calculating how many people lived there and what level of air quality they experienced.
The final step was to take this data, and, by using standard techniques in KCERG’s arsenal, calculate the resulting health impacts per grid square expressed in terms of changes in ‘life years’ and so life expectancy.
When the handle was finally turned, the conclusion that emerged was that ‘The population in Waltham Forest will gain around 41,000 life years, and increase life expectancy by around 1.5 months, if air pollution concentrations improve as projected to 2020, compared with remaining at 2013 concentrations’.
LBWF’s reaction was little short of triumphant. In its view, an independent and ‘world renowned’ research group had concluded that the ‘measures to prioritise pedestrians and cyclists…had made a marked contribution to improving air quality and health in the borough’. Better still, LBWF underlined, it was today’s young people who were forecast to benefit the most, with a Town Hall press release distilling KCERG’s somewhat ponderous conclusion into the attention grabbing and emotionally charged headline ‘Children’s life expectancy increases by six weeks thanks to…Waltham Forest’s road improvements’, which the Evening Standard turned into ‘Children will live longer thanks to Waltham Forest’s “Mini Holland” cycle scheme’, and other media outlets and campaigning groups then repeated.
Subsequently, LBWF’s claims have continued to generate attention and praise. Indeed, in 2019 LBWF won an Ashden Award, a ‘Green Oscar’, with its interventions to ‘clean up the borough’s air’ explicitly recognised in the citation.
Yet, such reactions notwithstanding, close scrutiny of the KCERG report, and some newly released correspondence, together raise doubts as to whether all is exactly as it seems.
It is worth noting to start with that the KCERG report is undated, and was written in something of a hurry, with the authors subsequently admitting, for example, that they did not analyse and incorporate LBWF raw traffic data ‘due to time constraint’. And while one senior LBWF politician, Cllr. Clyde Loakes, has asserted that the report was peer reviewed, there is no mention of that in its pages.
However, three further points are much more important.
First, it is clear that in some critical areas, KCERG depended not on its own observations and findings, but simply on what it was told by LBWF.
Thus, in the KCERG report’s discussion of how the school run impacts on air quality, it is admitted that LBWF supplied not only the ‘vehicle km’ (volume of traffic) data that such activity allegedly generated across the borough, but also a menu of scenarios documenting how the issue might develop, one of which included a very positive, though unsubstantiated, assessment of Mini-Holland’s likely impact.
Lack of time, again, may have played a part. But the fact that KCERG nowhere explained what protocol it had adopted in order to determine the reliability of LBWF data surely at the very least raises concerns. After all, LBWF was hardly a disinterested party.
Second, there are instances where KCERG’s analytical choices seem quite arbitrary. One example is the way it dealt with future traffic flows.
For reasons that are not spelt out, KCERG modelling started from the assumption that the total volume of traffic flows in Waltham Forest would remain unchanged between 2013 and 2020.
In passing, it is worth remarking that such an assumption is rather unexpected, implying as it does that for KCERG, at least, one of Mini- Holland’s main purposes, to stem motor vehicle use, already was likely a non-starter.
However, setting that aside, what KCERG certainly did accept was that there would be changes in the balance of the flows. Roads closed under Mini-Holland would see fewer motor vehicles. But this inevitably meant that roads which were not in Mini-Holland would cumulatively see more motor vehicles. The question, then, was how this increase should be distributed in terms of location on the ground (and therefore number of people affected) for 2020.
KCERG’s report is not very clear on this point, but when the lead author was later consulted, he advised that in each ‘1km by 1km grid’, those ‘minor/residential roads’ that were outside the Mini-Holland scheme ‘were adjusted upwards uniformly by an appropriate value’.
To a modeller, this might seem a plausible way of working, a sensible solution to a difficult problem, but to those familiar with the geography of the borough, the flaws are obvious. For by 2018, it was already apparent that increased traffic flows outside the Mini-Holland scheme, on what KCERG called ‘minor/residential roads’, were far from being evenly distributed. The classic case was Shernhall Street, the location of several schools, where, as a consequence of nearby road closures, traffic flows increased by 28 per cent (some 2000 vehicles per day), thus exposing many young people to considerably higher levels of air pollution than before.
The possibility, then, was that by averaging out traffic flows, in other words spreading them thinly across a plethora of locations, KCERG’s calculations underestimated the extent of roads with high air pollution in 2020, meaning that, for a truer picture, the associated health impact figures needed adjusting upwards, so in turn pushing the suggested gain in life expectancy downwards.
Finally, there is an issue right at the heart of KCERG’s basic methodology that is even more troubling.
KCERG’s model, to repeat, was modified to take into account both ULEZ and ‘other London policies’, and also ‘extra information specific to Waltham Forest, such as the Mini-Holland scheme’.
By definition, since the construction of the model was a numerical calculation, each of these factors must have been given a numerical value or weighting.
However, in its report KCERG nowhere documents how this was achieved.
And more puzzling still, when subsequently asked to explain, a KCERG team member stated:
‘Some effects [of the Council’s Mini-Holland interventions] were incorporated in the 2020 projections but it is difficult to estimate them and that’s why we cannot quantify mini-Holland contribution [sic]’.
That KCERG had included factors in its numerical calculations, yet subsequently admitted ‘it was difficult to estimate them’, obviously raises some very fundamental questions about the robustness of its conclusions, to the say the very least.
What makes the situation less palatable still is that, when Waltham Forest residents subsequently approached KCERG about these issues, the response was at best patchy.
One polite and reasonable questioner at first was provided with a certain amount of further information, true, but then the shutters came down. The tactics followed a well-trodden path: first a familiar litany of excuses (the issues raised were very technical, the relevant member of the KCERG was away, etc.), next prolonged silence, and finally that classic brush off, ‘we have moved on by several projects now and simply don’t have the time/resource to follow up with further information’.
As a market leader, no doubt KCERG is virtually always very busy. But universities are public institutions; the KCERG report was underwritten by local council tax payers; and no doubt KCERG’s involvement in Waltham Forest helped burnish its reputation.
All things considered, therefore, KCERG surely had a responsibility to be more forthcoming.
LBWF’s role in this story also merits closer scrutiny.
To start with, it is worth looking at why LBWF commissioned KCERG in the first place.
The sequence of events was as follows. During 2018, and in parallel with a public consultation, LBWF adopted an Air Quality Action Plan (AQAP) to cover the next five years.
In its original iteration, the AQAP included thirty-eight separate initiatives.
However, at the last minute, after the public consultation had ended, LBWF added a thirty-ninth measure, ‘Research Project with Kings College London’, which was justified on the basis that it would ‘Assist in communicating the health benefits of modal shift from car to cycling and walking’.
Why this late addition? The findings of the public consultation offer a clue, with LBWF presenting them so (the overwriting of labels appears in the original):
This was a time, of course, when LBWF viewed Mini-Holland as a flagship programme, indeed probably the one which was most likely to attract government plaudits and further funding. Yet what the evidence showed was that local residents were, to put it bluntly, not very interested, with less than a tenth of respondents mentioning Mini-Holland in any shape or form.
It is reasonable to speculate, then, that for some senior figures in the Town Hall, KCERG was commissioned primarily because it was an opportunity to address Mini-Holland apathy, and considerations about the intricacies of accurately measuring air quality were low down on the agenda.
However, by proceeding in this way, LBWF to some extent made a rod for its own back. The hope might be that Mini-Holland would emerge as a key driver of air quality improvement, but the reality was never going to be as clear cut. And this was simply because, as one expert commented at the time, ‘Air quality is influenced by a wide range of physical and environmental factors at any given time and it will be difficult to attribute changes over time to any one source of pollution or intervention’.
It is for this latter reason that, when KCERG reported, it was careful not to over-egg the pudding. The authors clearly approved of Mini-Holland – in the Evening Standard, one described it as ‘leading the way for healthier, less-polluted cities’ – but they did not anywhere suggest that it alone was responsible for the projected air quality improvements they modelled. Indeed, other possible determinants were either explicitly referenced from the first (as noted) or cited where appropriate. Thus, for example, deep in the KCERG report, there is the telling observation that the predicted ‘continuous declines’ from 2013 in local nitrogen dioxide concentrations were ‘likely to be mostly due to the improvement in…emissions of large parts of the road transport sector’.
In this situation, when KCERG finally presented its findings, LBWF was left with the choice of burying them, and possibly losing face, or giving them the best possible spin, and hoping that nobody read the small print.
To no long-term LBWF watcher’s surprise, spin won out, repeating what had happened on several other occasions (for a similar case, see the ‘knife crime’ and ‘Gang Prevention Programme’ links below).
The final lamentable aspect of this saga is that, as regards measuring how Mini-Holland impacted on air quality, LBWF always had other options open to it which almost certainly would have been more effective than commissioning a single report based upon theoretical modelling.
For example, LBWF could have installed relatively low cost sensors at the most crucial locations in the borough (such as outside schools) prior to motor vehicle restrictions being put in place, and then tracked how air quality fared subsequently, thus generating a fairly robust ‘before and after’ framework for comparison of hard evidence over time.
As is so often the case with LBWF, then, what happened is also, in the end, the story of a lost opportunity.
