I am glad that Gareth Jones wants the RPI CPI Group to study this issue. If I am not mistaken, the ONS already uses the Tukey method to identify outliers. This is well described in the UNECE-ONS manual A Practical Guide to Calculating Consumer Price Indices.
Unfortunately one of the drawbacks of the Tukey method is that it requires quite large price samples so I am not sure what, if anything, the ONS does in the case of smaller samples.
The Practical Guide also discusses one variant of the quartile method for outlier detection, employing the Hidiroglou-Berthelot transformation, named for two methodologists who work for Statistics Canada. It is used, unless things have changed, for outlier detection in the Danish CPI. At my insistence, it was also used to calculate outliers for the traveller accommodation CPI when it was redesigned based on a random sample. While the prouction system spit out a lot of outlier hotel price relatives these were never properly analyzed by any methodologist. I complained about this sad state of affairs to the director of Consumer Prices Division, Richard Evans, but as was his wont, he did nothing.
Since then, the thinking among methodologists at StatCan has changed, and when they were asked to recommend an outlier detection procedure for the CPI, they recommended the use of the quartile method with a log transformation of the data. As the Hidirioglou-Berthelot transformation approximates a log transformation, the difference is not huge.
This was supposed to be introduced in the new CPI production system but as far as I know, it never happened. Mr. Evans claimed that it didn't look like a go because of small samples. When I asked him if he thought our price samples were any smaller than those in the Danish CPI, he remained mute, as is also his wont.
A really sophisticated version of the quartile method could use a log transformation of the price data for one series, a different transformation or another, and I believe that StatCan's business survey methodologists had the prototype of a system that would do this when I worked there. It is likely much better developed now.
As you can see, my bias is towards the quartile method, which is about all I have ever worked with, in one variant or another, for about the last 30 years. I don't know as much about the Tukey method, which may be quite appropriate for larger samples, or other methods of interest.
Anyway, it is a fascinating subject, and I hope that Gareth is successful in getting the Group to look more into it. Unfortunately, it is really more a question for the methodologists in the group, rather than economists like me.
Show Original Message