

I actually did one further check of the data  a Tukey outlier correction method to the data supplied by the ONS (to be clear this is a different method to that described in the CPI manual). I am uncertain about the merits of this (an outlier correction of an outlier correction) but the resulting data that results had similar arithmetic and geometric means.
This tells me that the formula effect is, in practice, about the tails and how they are treated. It doesn't though say whether the tails are 'reasonable'. I am a bit wary of simply throwing extreme values away (they may reflect true costs which people face) but at the same time I am not sure that such extreme movements are truly likeforlike price variation. How much is sampling variation (the price collector picking a slightly different product can produce a lot of variance in itself)? How much is down to sales (and in which case are items of similar quality, etc.).
I appreciate these are very tricky judgments, but I would hope some discussion of these issues is touched upon in the ONS' onoing work surrounding price collection methods.
Show Original Message
 Original Message: Sent: 21032013 10:44 From: Gareth Jones Subject: Outlier treatment and the Median of Price relatives
I think there is still a degree of doubt about exactly what ONS does with outliers. Last year I attended a user group meeting where I asked a question about this. The reply I got from ONS was that outlier identification (the nature of which was not specified) was conducted in order to credibility check collected data. However, if the value was confirmed as correct, this "confirmed outlier" would still be used in whatever formula was being used for the index. This was not what I would call treatment of outliers. I think ONS needs to specify in the technical manual, exactly what it does do.
Your work on this subject already shows that the treatment of outliers is as important, if not more important, than the choice of formula, and the two issues cannot really be separated. This underlines the need for a comprehsive study of outlier treatment across all sectors and all methodologies.
Outlier treatment is inherently subjective. If one believes that an outlier is part of a representative sample and just increases the variance of estimates, there is no need for outlier treatment at all. If however, one takes the view that the sample is not representative (probably true) then some degree of outlier treatment is necessary. Too much treatment leads to bias one way, too little to bias the other way.
GJ
 Original Message: Sent: 12032013 13:59 From: Jonathan Gardner Subject: Outlier treatment and the Median of Price relatives
Andrew's email prompted me to contact the ONS to clarify the situation. The ONS does indeed use outlier techniques (the Tukey algorithm) to remove extreme values. They would have applied these techniques to the data supplied as part of the consultation. So my analysis will have been a double outlier correction / analysis. Hence the calculated figures could well be throwing away too much information, though I personally still find it interesting to see how much the calculated inflation rates are driven by the tails of the numbers that survive ONS trimming.
Gareth  I am not a fan of arbitrary rules (such as cutting price relatives below 0.67 and above 1.5) as I think you risk dropping too many true data points. Setting the rules can also imply strong beliefs on the data (in this case no sales or postsales price reversion in the index). For that reason I tend to favour statistical adjustments based on looking at the tails of the distribution.
Nevertheless I did test the approach Gareth suggested (dropping price relatives below 0.67 or above 1.5) . Generally it seemed to produce lower estimates of inflation (than the Jevons as currently published). This was particularly true for items with lognormal distributions of price relatives (and a long tail of highly positive price relatives).
Jonathan
 Original Message: Sent: 28022013 12:31 From: Andrew Baldwin Subject: Outlier treatment and the Median of Price relatives
I am glad that Gareth Jones wants the RPI CPI Group to study this issue. If I am not mistaken, the ONS already uses the Tukey method to identify outliers. This is well described in the UNECEONS manual A Practical Guide to Calculating Consumer Price Indices. Unfortunately one of the drawbacks of the Tukey method is that it requires quite large price samples so I am not sure what, if anything, the ONS does in the case of smaller samples.
The Practical Guide also discusses one variant of the quartile method for outlier detection, employing the HidiroglouBerthelot transformation, named for two methodologists who work for Statistics Canada. It is used, unless things have changed, for outlier detection in the Danish CPI. At my insistence, it was also used to calculate outliers for the traveller accommodation CPI when it was redesigned based on a random sample. While the prouction system spit out a lot of outlier hotel price relatives these were never properly analyzed by any methodologist. I complained about this sad state of affairs to the director of Consumer Prices Division, Richard Evans, but as was his wont, he did nothing.
Since then, the thinking among methodologists at StatCan has changed, and when they were asked to recommend an outlier detection procedure for the CPI, they recommended the use of the quartile method with a log transformation of the data. As the HidirioglouBerthelot transformation approximates a log transformation, the difference is not huge.
This was supposed to be introduced in the new CPI production system but as far as I know, it never happened. Mr. Evans claimed that it didn't look like a go because of small samples. When I asked him if he thought our price samples were any smaller than those in the Danish CPI, he remained mute, as is also his wont.
A really sophisticated version of the quartile method could use a log transformation of the price data for one series, a different transformation or another, and I believe that StatCan's business survey methodologists had the prototype of a system that would do this when I worked there. It is likely much better developed now.
As you can see, my bias is towards the quartile method, which is about all I have ever worked with, in one variant or another, for about the last 30 years. I don't know as much about the Tukey method, which may be quite appropriate for larger samples, or other methods of interest.
Anyway, it is a fascinating subject, and I hope that Gareth is successful in getting the Group to look more into it. Unfortunately, it is really more a question for the methodologists in the group, rather than economists like me.
 Original Message: Sent: 27022013 05:43 From: Gareth Jones Subject: Outlier treatment and the Median of Price relatives
I have recently posted a copy of chapter 1 of the ILO CPI manual in the library.
Paragraph 1.136 raises the issue of outlier treatment for the Jevons formula to reduce downward bias. Others have advocated outlier treatment for the Carli formula to reduce upward bias. In reality almost any formula used should be subject to outlier treatment
I suggest excluding all monthly price relatives greater than 1.5 or less than 0.67. This would remove the effects of short term half price offers which are not really part of the price trend but simply increase the variance (as well as creating bias).
One formula where this would not be necessary would be the median of price relatives. This formula is very robust to oddities in the data yet it has received virtually no attention in the literature or from ONS, although ONS uses the median in the ASHE survey of earnings.
I would like to see outlier treatment and the Median of Price Relatives form part of ONS' research programme before any more changes are proposed to indices and their formulae.
Of course we can forget about satisfying axioms under both of these approaches. They are not neat enough mathematically for that, but that is a reasonable price to pay for more accurate indices.
GJ


