You probably saw on the news the shocking headline that 40% of motorcycle users are riding around on untaxed bikes. The head of the House of Commons Public Accounts Committee (PAC) (download pdf of the PAC's Vehicle Excise Duty report here), Edward Leigh, went as far as to comment:

Large parts of the biking community are cocking a snook at the law.

Which would be fine if it bore any resemblance to reality. Anyone who is actually a part of the biking community will have probably been scratching their heads trying to work out who these evaders might be, as have our friends at MAG - they say:

Anecdotal visual studies carried out by the group at motorcycle events do not reflect anything remotely like this level of non compliance. (MAG article)

So what's going on?

Well a press release by the Motor Cycle Industry Association (MCIA) (download it in Word format here)questions some of the methodologies used by the government statisticians. David Taylor, head of the MCIA, says:

We are expected to believe that motorcycle VED evasion rose by 47 per cent from an already highly unlikely figure the previous year. Common sense suggests that the estimate of nearly 40 per cent is wildly inaccurate, or they would surely be very easy to catch.

That seems obvious enough, but what isn't at all obvious is the methodology and statistics adopted by the Department for Transport (DfT), who commissioned the traffic survey, the private company that carried out the survey, National Statistics who analysed it, DVLA who provided large amounts of data and the Commons PAC (the group of MP's who have to interpret all this), who caused all the fuss.

Now there's no requirement to be trained in statistics if you are an MP on the PAC, so despite Mr Leigh's intemperate outbursts against bikers, we can't blame him or the rest of the committee for taking the DfT report at face value.

When you take a deeper look at the DfT report and the National Statistics report that underlies it, though, it's obvious that there are some big assumptions regarding the data and the statistical techniques that have been carried out on it.

This might get quite heavy, but bear with me - there's not too much maths.

Now I'm assuming that not many people reading this have any kind of statistics qualification, so I'll try to summarise what I've been able to work out without using too much maths. Unfortunately, the relevant bodies above haven't been that kind, so if anyone is better at this kind of maths than me here are the original documents, feel free to post comments:

1. National Statistics VED stats 2006
2. National Statistics / DfT report of above VED stats.(pdf)
3. Statistical Review of the VED figures by Southampton University

In what follows I'll refer to these documents by the number I've given each one.

Looking at the full published figures, you will find hidden away in an appendix a list of confidence ranges for the final estimate of percentage of vehicles with no VED (road tax). (1 - appendix E7 / Table 18) Confidence range is a statistical term, but it's not that hard to grasp, using the figure in question as an example. What it means in practice is that although the average evasion recorded is running at 37.8 percent, this is an estimate, but based on the data collected they can say they are 95% confident that assuming all their prior assumptions are true the true figure lies somewhere between 29.9% and 45.7%. This is a truly massive margin for error (for comparison the equivalent 95% confidence figures for cars are 4.0%-4.6%) and shows that even in the best case scenario the margin for error in the bike figures is going to be running at ±20%.

Why is this figure so massive. Well, there are a few reasons, but it mainly comes down to the fact that the number of bikes counted in the survey was much, much smaller than the number of cars. For every bike they counted, they saw 110 cars. (1) Table 10 Whenever you have a small sample, your uncertainty will be larger. Small uncertainties in a sample also tend to balloon when you perform other operations on that sample, as you introduce extra uncertainties which multiply through.

As I mentioned in passing above, all of these error margins are "best case scenarios" and rely very much on the assumptions made by the method used to derive the figures. If there are incorrect estimates made due to these assumptions the final figures will be seriously distorted

For this survey I believe that the assumptions made are in many cases entirely wrong, and I believe this has played a large part in inflating the figures. And although the figures have been independently checked by statisticians those doing the checking have done so on the basis that these assumptions are true (3), as they rightly state at the start of the analysis.

Assumption (b) is a crucial one. In statistical language it is this:

the observed sample of vehicles sighted in the Roadside Traffic Observation Survey is a simple random sample with replacement of the registered vehicles(3)p13

This is not immediately obvious, and uses technical terms, but what it means is this. Every time the person at the side of the road takes a measurement, the chance of seeing a particular vehicle pass by is the same as if he were picking registration numbers at random from a massive lottery machine containing a single ball for each vehicle in Britain. This is justifiable, if you were to be recording the traffic on every road in Britain, but in practise, with only 249 sites, this method has potential to be badly skewed. Anyone who happens to live near one of the sampling sites has a much higher probability of being 'picked' (and probably picked multiple times at that) than someone who never passes by.

What this means is that the selection of sample sites is going to have a large bearing on what is recorded. The sample sites chosen represent 1 of each kind of road (as defined by the DfT) per police force area (49 of those) with London getting an extra 3 of each. Motorways are sampled by local government region, and fewer of those are picked.

How have they selected which roads to measure? Well they left that "to the discretion of contractors" who had to reach a set minimum number of vehicle sightings at each location. We can guess they probably chose fairly busy examples of each type of road. It seems likely that the type of person who is going to dodge road tax is more likely to be of a lower social class than average, and live in a scummier area, probably closer than average to a busy road. This is supposition, but it is reasonable, and there is nothing in the results the DfT have presented to suggest that this kind of sampling bias has been effectively eliminated.

Another assumption taken by the survey is that stated quite clearly in (3)

One of the most important assumptions in the model is that the average number of sightings of a given vehicle is proportional to its mileage. This hypothesis is not testable from the survey data itself because the mileage of individual vehicles is not directly observed through the survey process. However, the first time that this working assumption was adopted - see §4 in Appendix C of (Department of Transport, 1984) – a postal survey of the keepers of heavy goods vehicles was used to test the adequacy of this hypothesis. Given that this research was carried out some time ago and for a limited sample of vehicles in a single tax class, the Department for Transport should investigate whether alternative data sources exist, or could be obtained, which could be used to re-examine the validity of this crucial assumption.

Or, put differently. it seems unlikely that motorcycles on the road today are being used in the same way as trucks were used in 1984! Estimated mileages for bikes are therefore likely to be way out of kilter with actual figures.

All of these assumptions seem likely to overestimate the proportion of bikes appearing to evade road tax, but there is potentially a far bigger problem, which is not even mentioned in any of the DfT documentation - bikes are much more likely than the average vehicle to be taken off the road.

Many bikers, as we all know (but perhaps the DfT doesn't) are fairweather bikers. Many more people own a bike but might, like my Dad, keep it locked in a garage for years at a time. Many of these people will have notified the DVLA that the bike is stored off-road via a SORN form, but I would guess that a lot of people don't. We probably all know someone with a bike in their garage that they probably didn't use at all last year.

Notice there are two different figures for motorbikes without VED. The figures are 16% of motorbikes in traffic, and 37.8% of vehicle stock (i.e. all bikes with a reg number). Why is the second figure more than double. The logic runs like this. The vehicles they spot on the road tend to be those that travel more miles, so they will have not counted lots of vehicles that have a fairly low mileage. Because they also know that untaxed vehicles have lower average mileages, they apply a corrective figure.

What they don't seem to have taken account of, though, is that at any given time, a very large number of motorcycles are sat in a garage not being used for months at a time, hence, for those bikes with a mileage of 0, they may be estimating high levels of tax avoidance!

If the assumptions about motorbike usage are corrected, we might find that this doubling effect will vanish.

But there is one last doubt I have about these figures, that might potentially show massive levels of VED avoidance where very little exists.

It's to do with the way the survey has been carried out and the figures derived.

All of the survey data was carried out in June and July of 2006, a time of the year when a lot of bikes are on the road. At some point the registration numbers queried were tested against the DLVA's VED database. The survey results were first published on January 25th 2007. Following that a review was carried out on the figures, and the PAC finally got round to studying them just a few days ago.

The crucial question is this - when were the data compared against the DVLA database, and what figures did they use?

The survey contractors almost certainly didn't have the means to check the DVLA database in real time, and probably didn't have access to the data, anyway. It seems likely that they would have returned all the data at once, at the conclusion of the survey period. But they probably carried out their own checks on the data before they did. So it seems likely that some time between July 06 and January 07 they checked against DVLA records. It then takes the DfT and National Statistics a further six months to process the data - they probably have a lot of validation and checking work to do, but exactly what and in what order we do not know.

So consider the following scenario. I get my bike out of storage on March 1st and tax it for six months. During June I ride by one of the government surveys and am counted as "on the road" and "in traffic". Come August 1st the bike is back in the garage and SORNed for the winter. If the DfT didn't get round to checking the VED database until September I'll probably be recorded as not being taxed.

Did this happen? I don't know. But realistically, the a huge number of bikers are going to put their bikes in storage through the winter and claim a VED refund by way of SORN, so if it did, the effect could be a huge overestimate of the number of bikers dodging tax.

Again I don't know if this has happened, but I have asked National Statistics for more detail on how the VED status was derived and when (if) they reply, I will post here.

Well, this is a mammoth post already, but we've reached the end. There are other areas where an unintended bias may have crept into the sample, but we've considered what I suspect are the biggest sources of potential error.

The bottom line is that I think it is extremely unlikely that as many as 40% of bikers are evading VED and what this shows more than anything is the danger of placing total faith in your statistics when the underlying assumptions are not a realistic model of the situation you are trying to assess, as well as the difficulty of designing a survey.

And the motorbike insurance angle, of course, is that all of these people without tax presumably have no insurance. Given the prevalence of Automatic Numberplate vehicles run by the police forces now, you'd think they would have noticed if 40% of the motorbikes going by were uninsured, wouldn't you?



************************************UPDATE***************************

I've received my reply from National Statistics to my query about when the survey data was checked against the DVLA database.

Here is the text of the email:

David,

Thank you for your enquiry.

The data for the VED survey in June 2006 was checked against the DVLA system in September 2006 to allow for late updates to be made.

P*** S****
www.dft.gov.uk/transtat/vehicles

So as I guessed above, anyone who SORNed their taxed bike before or during part of September will have been assumed to be dodging tax. This is going to have had a massive impact on the figures as reported in the press and the true figures for uninsured riding are much smaller than the ones the government has taken at face value.

Labels: big brother, bike, dvla, motorbike, motorcycle, police, road tax, sorn, uninsured, ved

link | posted by Dave Wilson at Friday, January 25, 2008
3 comments links to this post

It seems that these days, it's hard to find a policeman when you want one. But here at Bikesure, we've found what we think is the fastest and easiest way to distract a copper from his latest doughnut and turn up at your house.

Yes, simply upload a video of your bike travelling at high speeds and overtaking dangerously to your favourite video sharing site, and a member of Her Majesty's Constabulary will be despatched to take a statement, as soon as they've conducted a thorough and wide-ranging nationwide investigation.

That's what this chap did,



and now he simply has to wait for the police to complete their investigations into who he in fact is, and his next knock on the door will likely be the entire South Yorkshire force.

Say the police:

"We will do everything we can to identify these riders."

Seriously though:

Bikers: Don't ride like a muppet (that overtaking was shocking), and if you do, don't film yourself doing it, and if you must, don't post the film onto the internet.

Police: Here's a free tip: instead of scouring the internet looking for dangerous road users, who you then can't find without a lengthy and extensive investigation, why not look for them on the roads, where they actually are, in real life, and what's more you might actually catch and stop them in the act?

Labels: bike insurance, biker, biking law, motorcycle law, police, speeding, video

link | posted by Dave Wilson at Monday, September 03, 2007
1 comments links to this post

Hallelujah, at last someone in our draconian police force has seen the light (forgive the pun). According to a new police agreement you will now not be prosecuted if you are wearing a dark visor. How many times have you or your mates been stopped on a bright summers day because you’ve got a dark visor on your expensive safety equipment (which you have to wear by law); while the boy racers drive by in their hot-hatch cars with exhausts that an illegal immigrant and his family could live in; with windows that are as dark as the inside of a coalminers nether regions, playing their music so loud it could be heard on the next continent.

David Griffin head of Motorcycle Safety for the Association of Chief Police Officers (ACPO) who is also Deputy Chief Constable for Humberside brought together members of the police community from around the country who had similar views have now decided that instead of being prosecuted it is up to the officer to use their common sense, so come on, if you have a dark visor on at night it’s quite right that you get pulled up.

Also illegal number plates will be judged on an individual basis, as long as they can be read from 20.5m you will be OK (so watch what script you choose for your personal plate).

These recommendations only apply in England, for our friends north of the border these do not apply. BUT a spokesperson for the Association of Police Officers in Scotland (APOCS) said “We are allowing our officers to use their own discretion”, so next time you get stopped, be nice and they might just let you off with a cautionary piece of advice,

Thanks to the MCN for bringing this innovative decision to our attention. What will be next, perhaps polite considerate Volvo drivers, Na that we be too much!

Don't forget that if you get yourself an expensive helmet and leathers you can add full cover for your accessories to your Bikesure policy for only £20. Get a quote online.

That’s all for now stay safe,

1leggedfreak.

Labels: bike, helmets, police, tinted, visor

link | posted by Dave Wilson at Thursday, April 05, 2007
0 comments links to this post