I think it would be best to avoid using the terms outlier and bias here. We don't know the quantities represented by each price relative, so we cannot calculate a bias.

What is true is that the AM is more responsive than the GM to high price relatives and less responsive to low ones. None of the values is necessarily an outlier. The nature of the sample (PPS or otherwise) is very important here.

Original Message:

Sent: 28-04-2017 03:06

From: Jill Leyland

Subject: Skewness and geometric and arithmetic means.

Dear all

This has been an extremely interesting and useful discussion to follow and one that will certainly help me the next time I am involved in any discussions about the formula effect (including when it is raised in the Stakeholder Panel).

Michael's suggestion of plotting the AM and GM of (1,1,1,1,1,x) against x illustrates the biases in Carli and Jevons nicely and is an approach I can see myself using the next time I am trying to explain the issue.

Many thanks to all those who have posted and I look forward to any future posts.

------------------------------

Jill

------------------------------

-------------------------------------------

Original Message:

Sent: 27-04-2017 13:23

From: John Wood

Subject: Skewness and geometric and arithmetic means.

I've been following this discussion with interest and Arthur's latest posting has prompted me to make a few comments.

From my own experience of examining price relative distributions, the most distinguishing feature I found was not skewness or multi-modality but extreme leptokurtosis. This is because, over relatively short time periods, the vast majority of prices do not change at all, producing a dominant spike in the distribution at 1. This is supported by almost all of Arthur's graphs (although the spike is strangely labelled as at 0.99, presumably because of the automatic scale generator). The one exception is for baked beans in 2005, where the spike is around 1.15 or thereabouts, presumably because of a general increase in bean prices over that year (although there is still a second, smaller spike at 1).

Any distributional approximations that don't allow for this leptokurtosis are likely to be misleading. I have seen this leptokurtosis remarked on in a few internal ONS papers, which are now presumably lost in the archives. It seems that ONS staff, like the rest of us, need to perpetually relearn the lessons of history. Perhaps we should require all price index statisticians to learn about the leptokurtic nature of price relatives.

We did try to approximate price relative distributions using a mixture of two Normal distributions, one tall and skinny, the other short and fat. This could replicate the general shape of these distributions but the approach was never completely satisfactory, not least because of the discrete nature of these distributions, as Arthur has noted. This arises because producers tend to increase prices in occasional steps rather than in daily infinitesimal increments. This is one reason why the Divisia index is not a good model for price indices.-------------------------------------------

Original Message:

Sent: 25-04-2017 05:17

From: Arthur Barnett

Subject: Skewness and geometric and arithmetic means.

Many thanks to ONS for providing the links to the historic data so quickly which allows more charts to be presented.

I have repeated the two December 2016 charts for 2014 – the year Rupert de Vincent-Humphreys' paper uses – and for 2005 which is prior to the change to fashion data collection and also before the financial crisis.

The story remains broadly the same in 2005 and 2014 as in 2016. The two additional years add weight to the argument that price relative distributions may be multimodal unlike the normal or log-normal. The multimodality of the baked bean price relatives moves around as would perhaps be expected depending on the rate and direction of price change. The women's vest/strappy top price relative distributions are less clear but again display clustering. The 3 fashion charts appear to show an increasing range and right skew of price relatives over time though 3 months data is insufficient to draw any conclusions. The women's vest/strappy top price relative distributions may indicate that the issues are more with the data collection methodology than the formulae.

Problems with using the normal and log-normal distributions as the basis for estimation were acknowledged in an ONS article from 2012 "Stochastic and Sampling Approaches to the Choice of Elementary Aggregate Formula" – https://www.ons.gov.uk/economy/inflationandpriceindices/methodologies/pricesuserguidancemethodologyanddevelopments

__ __

This article looked at the use of the Normal (arithmetic mean) and Log-normal (geometric mean) distributions as the basis for averaging price relatives and states –

__ __

"Statistical tests did not find evidence that either of the distributions fitted the data well. However, two-thirds of the elementary aggregates were closer to a log-normal than a normal distribution, indicating that for these a Jevons may be preferred."

The article did not consider multimodality and therefore the second sentence quoted above may simply reflect the asymmetry of the log-normal distribution and price relatives whereas the normal is symmetric.

It will be interesting to see if the forthcoming article will look at a wider range of approaches than the computationally simple normal and log-normal that were the subject of the 2012 article.

The 2014 charts –

All items Dec/Jan inflation was lower in 2014 than 2016 (CPI 1.2% compared with 2.3%, and RPI 1.9% compared with 3.2%). The baked beans price relative again appears multimodal but in 2014 is left skewed. There is also another cluster of 16 with a price relative of 0.61. The women's vest/strappy top chart in 2014 is similar to that in 2016. It is more compact than in 2016 with a narrower range that is closer to symmetrical with in relative terms – times 4.33 to the left and 5.33 to the right. There remain apparent clusters at 1.25, for example.

The 2005 charts –

All items Dec/Jan inflation is similar for both CPI and RPI in 2005 at 2.6% and 2.8% respectively. The 2005 baked bean chart again appears at least bimodal with the largest clusters not at a price relative of 1 but of 1.6 and 1.7. The women's vest/strappy top chart in 2014 is similar in shape to that in 2014 and 2016. It is more compact with a narrower range than in 2014 that is even closer to symmetrical in relative terms – times 3 to the left and 3.45 to the right. There remain apparent clusters at 1.15, for example, but with the sample size two thirds of those for 2014 and2016 it is more difficult to pick out patterns..

All the best.

Arthur

**Links to price relative data**

https://www.ons.gov.uk/economy/inflationandpriceindices/datasets/consumerpriceindicescpiandretailpricesindexrpiitemindicesandpricequotes - latest data

http://webarchive.nationalarchives.gov.uk/20160105160709/http://www.ons.gov.uk/ons/guide-method/user-guidance/prices/cpi-and-rpi/cpi-and-rpi-item-indices-and-price-quotes/index.html - recent historic data

http://webarchive.nationalarchives.gov.uk/20160105160709/http:/www.ons.gov.uk/ons/rel/cpi/consumer-price-indices/cpi-and-rpi-item-indices-and-price-quotes/rpt-cpi---rpi-item-indices---price-quotes.html - older historic data

-------------------------------------------

Original Message:

Sent: 23-04-2017 10:18

From: Arthur Barnett

Subject: Skewness and geometric and arithmetic means.

There may be issues with Rupert de Vincent-Humphreys' paper but the analysis is grounded in actual price quote data which is to be welcomed. The paper uses December 2014 data but for this post is restricted to December 2016 data in part because 2014 data could not easily be found on the ONS web site.

Rupert de Vincent-Humphreys' paper uses fashion data which is a particularly difficult area to interpret. In contrast this post starts with the 2016 Q4 data for a more mundane product – baked beans – which should be better behaved. Tins of baked beans are a staple non-seasonal foodstuff with limited quality change issues and so would hopefully be well behaved in terms of price quotes data.

The following chart gives the frequency distribution for December 2016 –

The frequency distribution does not look like an iid sample drawn from either a normal or a log-normal distribution. This is perhaps not surprising as the two theoretical distributions are continuous defined on the real numbers whereas the price relatives distribution is discrete and based on only a limited subset of the rational numbers. This limited subset will be a product of, for example, the arithmetic of the small price range of tins of beans, the price collection methodology and the pricing strategies of retailers and manufacturers.

The range of values that baked beans price relatives take is from 0.81 to 1.58. There is a question of whether these extreme values may be unrepresentative outliers and can be altered or removed from the sample. This may be the case for the single 0.81 value but there are 5 price relatives at 1.58 and 32 at 1.5 - these latter two price relatives point to the possibility of the distribution of price relatives not being a single distribution but a mixture of two or more distributions. The simplest such mixture of distributions is the bimodal normal but although that may represent a better approximation than the normal or log-normal in this case it could still be problematic. HHH

Rupert de Vincent-Humphreys' paper was, of course, not looking at a well behaved product but looking at fashion which is reputedly not at all well behaved. A key chart from the paper is –

One question is whether the price quote data give any clues as to what may be causing this behaviour of the price indices post 2010.

The 2016 December price relatives for women's vests/strappy tops have a range from 0.199 to 10.179 and there is only one of each of these extreme observations though there are other observations that could be considered to be relatively close. This range gives a factor of 5 in the left skewness and of 10 in the right skewness – note this is not a statistical measure of skewness. The geometric mean is less robust to a large outlier to the left and the arithmetic mean is less robust to a large outlier to the right. This 2016 data has apparent outliers in both directions which if this is typical of price behaviour for this product since 2010 could help provide an answer to the question implicit in the comment in Rupert de Vincent-Humphreys' chart above –

"Observed prices increased more than CPI; much less than RPI."

Unfortunately the very large range for 2016 December women's vests/strappy tops price relatives means displaying a full frequency chart is impractical – it would need to be 10 times wider than the above baked beans frequency chart. The following women's vests/strappy tops chart uses the same vertical and horizontal scales used for the baked beans chart to allow some comparison across the different products –

The pattern again doesn't seem to resemble an iid sample from a normal or log-normal distribution. There appear to be small clusters around increases of 10%, 20% and 33% which perhaps points to retailer pricing policies and may indicate a multimodal distribution. The second largest frequency is outside the chart area at a price relative of 2 with a count of 17 which further suggests a multimodal distribution related to retailer pricing policies.

To conclude more research looking at the suitability of different averaging formulae for different individual products or services is needed. Arguably this should not start with the most difficult products or services but look at those that would be expected to be relatively well behaved. When an understanding of the better behaved products has been achieved then the issues with the more difficult products or services can be considered though those issues may primarily be with other parts of ONS price index methodology.

My understanding is that historically the choice of different formulae for the RPI was derived from looking at the properties of the data but this was done a long time ago so is unlikely to reflect the current behaviour of the public or businesses that serve them. I seem to remember that Andrew Lydon posted some time ago suggesting that such an exercise should be repeated. There is an opportunity to do this in the development of the household index – an opportunity it is to be hoped ONS will take.

Also given the computing power that is currently available – which was not available to the designers of the RPI – ONS could consider more data related approximations than the normal or log-normal distributions.

All the best.

Arthur

-------------------------------------------

Original Message:

Sent: 20-04-2017 11:14

From: Andrew Baldwin

Subject: Skewness and geometric and arithmetic means.

Dear Michael:

I don't dispute what you say. Neither, I am sure, would Mick Silver or his co-author. If you look at my paper "Common Sense Favours the Use of the Jevons Formula" I discuss the treatment of outliers on p. 9-12. Table 2 shows a simple example of a sample of four prices, where all prices are 100 except for one outlier value. For a low outlier, say 10, the Carli index is 77.5 and the Jevons index 56.234, so the Carli index is just 27% greater than the Jevons index. For the corresponding high outlier, 1000, the Carli index is 325 and the Jevons index is 177.828, so the Carli index is 82.8% greater than the Carli index. The differences get even more dramatic if one looks at more extreme low and high outliers. By assumption, the average price is assumed to be 100 so the closer the estimates are to 100 the better. To paraphrase Milton Friedman, the Jevons formula does better by low outliers than the Carli formula does by high outliers, even if it does worse by low outliers than the Carli formula. This is what I mean by saying it is more robust in the presence of outliers than the Carli formula.

Of course it would be better to identify and adjust for high and low outliers rather than falling back on the elementary aggregation formula to handle such problems, but this is not always possible. You may be interesting in knowing that the quartile method was originally proposed for outlier detection in the Canadian industry product price indices in the 1980s without any transformation of the price relatives. It was discovered that this tended to ignore low outliers, which was why StatCan methodologists recommended much later in 2008 that the quartile method be applied with a logarithmic transformation of price relatives in outlier detection for the Canadian CPI. See the reference to the 2008 paper by Saad Rais in my paper. The Danish CPI uses the quartile method with a Hidiroglou-Berthelot transformation of price relatives. This is very similar to the log transformation, and serves the same purpose of making sure that low outliers are not ignored. Unfortunately, as far as I know, the quartile method has not been adopted in a systematic way in either the industry product price indices or the consumer price indices. The methods that were used when I worked there were decidedly ad hoc, and likely remain so.

Thank you, by the way, for your analysis of the variance of the different formulas, which was very well done.

Best regards,

Andrew