Rupert de Vincent-Humphreys' paper APCP-T(17)02 "RPI and CPI: a tale of two formulae" includes –
"The difference between the RPI (Carli formula) and CPI (Jevons formula) results reflects, in essence, the difference between the arithmetic and geometric mean of a sample, which in turn is proportional to the variance of the sample."
The variance of the sample does not tell the whole story. The skewness – or asymmetry – of the sample is another important factor whose effects on how the different formulae behave need to be explored.
I attach a set of simple illustrative examples looking at the relative properties of the arithmetic and geometric means in left and right skewed and symmetric contexts. The examples are artificial and were chosen for ease of calculation in Excel.
The first two charts below show that the same pattern of increasing variances can result in very different behaviour of the formulae depending on the direction the sample is skewed.
The right skewed example shows what would be expected intuitively with the rate of change for the arithmetic mean being faster than that for the geometric mean with increasing variance. In contrast, and perhaps counterintuitively, the left skewed example shows the reverse with increasing variance the rate of change of the geometric mean is much faster than the arithmetic mean– see charts below and attached spread sheet.
The symmetric example, which is not skewed and has larger variances than the skewed examples, shows a constant arithmetic mean as would be expected, but the rate of change for the geometric mean is approaching that of the left skewed example.
Out of interest I also substituted the geometric mean for the arithmetic mean in the standard variance formula – I'm not sure if that is in any sense statistically meaningful . For these examples the geometric mean version is consistently marginally larger than the standard variance calculation. There are, of course, no standard Excel functions for the new version but I think that the calculation is ok.
This is not at all straightforward, of course, but in the terminology of Rupert de Vincent-Humphreys' paper the geometric average may be less robust than the arithmetic average in circumstances of left skewed and symmetric distributions of price relatives.
The penultimate bullet point in APCP-T(17)03 "Re-addressing the formula effect" is potentially helpful here providing the answer to the question it poses is yes –
"Should we be trying to provide a better understanding of the divergences?"
Could ONS confirm that in the forthcoming paper they will be pursuing this question in its widest sense including the mathematics of averaging in addition to sampling, quality change and observational error issues.
All the best.
Arthur's calculation is correct. However, in practice there is usually a large positive skew. If we can approximate the distribution by a lognormal such that log X ~ N(m, s²) then the GM is exp(m) and the AM is exp(m+s²/2) so AM/GM is exp(s²/2). The variance (var) is exp(m+s²)(exp(s²)-1) = AM²(exp(s²)-1) =AM²((AM/GM)²-1). If AM = 1 for simplicity, then var = (1/GM²-1) so GM = 1/sqrt(v+1).
What Arthur's conclusion shows is that the GM is more robust to a large outlier to the right but the AM is more robust to a large outlier to the left. This is easily shown by plotting the AM and GM of say (1, 1, 1, 1, 1, x) as a function of x.
Andrew's note raises an interesting point. He quotes the claim that "The geometric mean is known to be more robust to outliers." As I said yesterday, it is more robust than the AM to right-hand outliers but less robust to left-hand ones, which do also occur. Attempts to clean the data by removing or correcting the outliers may concentrate too much on right-hand outliers. When the GM is used, it is important to pay equal attention to left-hand ones to avoid a downward bias in the index.
There may be issues with Rupert de Vincent-Humphreys' paper but the analysis is grounded in actual price quote data which is to be welcomed. The paper uses December 2014 data but for this post is restricted to December 2016 data in part because 2014 data could not easily be found on the ONS web site.
Rupert de Vincent-Humphreys' paper uses fashion data which is a particularly difficult area to interpret. In contrast this post starts with the 2016 Q4 data for a more mundane product – baked beans – which should be better behaved. Tins of baked beans are a staple non-seasonal foodstuff with limited quality change issues and so would hopefully be well behaved in terms of price quotes data.
The following chart gives the frequency distribution for December 2016 –
The frequency distribution does not look like an iid sample drawn from either a normal or a log-normal distribution. This is perhaps not surprising as the two theoretical distributions are continuous defined on the real numbers whereas the price relatives distribution is discrete and based on only a limited subset of the rational numbers. This limited subset will be a product of, for example, the arithmetic of the small price range of tins of beans, the price collection methodology and the pricing strategies of retailers and manufacturers.
The range of values that baked beans price relatives take is from 0.81 to 1.58. There is a question of whether these extreme values may be unrepresentative outliers and can be altered or removed from the sample. This may be the case for the single 0.81 value but there are 5 price relatives at 1.58 and 32 at 1.5 - these latter two price relatives point to the possibility of the distribution of price relatives not being a single distribution but a mixture of two or more distributions. The simplest such mixture of distributions is the bimodal normal but although that may represent a better approximation than the normal or log-normal in this case it could still be problematic. HHH
Rupert de Vincent-Humphreys' paper was, of course, not looking at a well behaved product but looking at fashion which is reputedly not at all well behaved. A key chart from the paper is –
One question is whether the price quote data give any clues as to what may be causing this behaviour of the price indices post 2010.
The 2016 December price relatives for women's vests/strappy tops have a range from 0.199 to 10.179 and there is only one of each of these extreme observations though there are other observations that could be considered to be relatively close. This range gives a factor of 5 in the left skewness and of 10 in the right skewness – note this is not a statistical measure of skewness. The geometric mean is less robust to a large outlier to the left and the arithmetic mean is less robust to a large outlier to the right. This 2016 data has apparent outliers in both directions which if this is typical of price behaviour for this product since 2010 could help provide an answer to the question implicit in the comment in Rupert de Vincent-Humphreys' chart above –
"Observed prices increased more than CPI; much less than RPI."
Unfortunately the very large range for 2016 December women's vests/strappy tops price relatives means displaying a full frequency chart is impractical – it would need to be 10 times wider than the above baked beans frequency chart. The following women's vests/strappy tops chart uses the same vertical and horizontal scales used for the baked beans chart to allow some comparison across the different products –
The pattern again doesn't seem to resemble an iid sample from a normal or log-normal distribution. There appear to be small clusters around increases of 10%, 20% and 33% which perhaps points to retailer pricing policies and may indicate a multimodal distribution. The second largest frequency is outside the chart area at a price relative of 2 with a count of 17 which further suggests a multimodal distribution related to retailer pricing policies.
To conclude more research looking at the suitability of different averaging formulae for different individual products or services is needed. Arguably this should not start with the most difficult products or services but look at those that would be expected to be relatively well behaved. When an understanding of the better behaved products has been achieved then the issues with the more difficult products or services can be considered though those issues may primarily be with other parts of ONS price index methodology.
My understanding is that historically the choice of different formulae for the RPI was derived from looking at the properties of the data but this was done a long time ago so is unlikely to reflect the current behaviour of the public or businesses that serve them. I seem to remember that Andrew Lydon posted some time ago suggesting that such an exercise should be repeated. There is an opportunity to do this in the development of the household index – an opportunity it is to be hoped ONS will take.
Also given the computing power that is currently available – which was not available to the designers of the RPI – ONS could consider more data related approximations than the normal or log-normal distributions.
Many thanks to ONS for providing the links to the historic data so quickly which allows more charts to be presented.
I have repeated the two December 2016 charts for 2014 – the year Rupert de Vincent-Humphreys' paper uses – and for 2005 which is prior to the change to fashion data collection and also before the financial crisis.
The story remains broadly the same in 2005 and 2014 as in 2016. The two additional years add weight to the argument that price relative distributions may be multimodal unlike the normal or log-normal. The multimodality of the baked bean price relatives moves around as would perhaps be expected depending on the rate and direction of price change. The women's vest/strappy top price relative distributions are less clear but again display clustering. The 3 fashion charts appear to show an increasing range and right skew of price relatives over time though 3 months data is insufficient to draw any conclusions. The women's vest/strappy top price relative distributions may indicate that the issues are more with the data collection methodology than the formulae.
Problems with using the normal and log-normal distributions as the basis for estimation were acknowledged in an ONS article from 2012 "Stochastic and Sampling Approaches to the Choice of Elementary Aggregate Formula" – https://www.ons.gov.uk/economy/inflationandpriceindices/methodologies/pricesuserguidancemethodologyanddevelopments
This article looked at the use of the Normal (arithmetic mean) and Log-normal (geometric mean) distributions as the basis for averaging price relatives and states –
"Statistical tests did not find evidence that either of the distributions fitted the data well. However, two-thirds of the elementary aggregates were closer to a log-normal than a normal distribution, indicating that for these a Jevons may be preferred."
The article did not consider multimodality and therefore the second sentence quoted above may simply reflect the asymmetry of the log-normal distribution and price relatives whereas the normal is symmetric.
It will be interesting to see if the forthcoming article will look at a wider range of approaches than the computationally simple normal and log-normal that were the subject of the 2012 article.
The 2014 charts –
All items Dec/Jan inflation was lower in 2014 than 2016 (CPI 1.2% compared with 2.3%, and RPI 1.9% compared with 3.2%). The baked beans price relative again appears multimodal but in 2014 is left skewed. There is also another cluster of 16 with a price relative of 0.61. The women's vest/strappy top chart in 2014 is similar to that in 2016. It is more compact than in 2016 with a narrower range that is closer to symmetrical with in relative terms – times 4.33 to the left and 5.33 to the right. There remain apparent clusters at 1.25, for example.
The 2005 charts –
All items Dec/Jan inflation is similar for both CPI and RPI in 2005 at 2.6% and 2.8% respectively. The 2005 baked bean chart again appears at least bimodal with the largest clusters not at a price relative of 1 but of 1.6 and 1.7. The women's vest/strappy top chart in 2014 is similar in shape to that in 2014 and 2016. It is more compact with a narrower range than in 2014 that is even closer to symmetrical in relative terms – times 3 to the left and 3.45 to the right. There remain apparent clusters at 1.15, for example, but with the sample size two thirds of those for 2014 and2016 it is more difficult to pick out patterns..
Links to price relative data
https://www.ons.gov.uk/economy/inflationandpriceindices/datasets/consumerpriceindicescpiandretailpricesindexrpiitemindicesandpricequotes - latest data
http://webarchive.nationalarchives.gov.uk/20160105160709/http://www.ons.gov.uk/ons/guide-method/user-guidance/prices/cpi-and-rpi/cpi-and-rpi-item-indices-and-price-quotes/index.html - recent historic data
http://webarchive.nationalarchives.gov.uk/20160105160709/http:/www.ons.gov.uk/ons/rel/cpi/consumer-price-indices/cpi-and-rpi-item-indices-and-price-quotes/rpt-cpi---rpi-item-indices---price-quotes.html - older historic data
Your point about the 2005 baked beans chart is interesting in a nerdish sort of way – sadly I ended up knowing too much about the price of beans because the Johnson Review at one point seemed to be about to use them as an example. The beans for baked beans are imported from the US and Canada and will almost certainly be sold in US$. The pound was strong against the dollar around 2005 and so the rise in price was probably due to a supply constraint – harvests fluctuate and with them prices. There was another steeper increase in price at about the time of the financial crisis and that was probably mainly the result of a fall in the pound though if I remember rightly ONS changed the collection of baked beans about that time. It will/might be interesting to see if the price of baked beans increases in 2017 with the fall in the pound following the Brexit referendum and what that does to the distribution of price relatives.
You also make a good point about leptokurtosis which adds another interesting dimension to the problem of the distribution of price relatives. Its value is likely to be in exploring further approximations to the distribution of price relatives among continuous probability models. Identifying continuous approximations tends to rely on grouping discrete observations. This can hide the detailed structure of the price relative distribution as can be seen from the chart below for December 2016 women's vests/strappy tops which has been grouped unlike the first set of charts.
The distribution presented in this way shows few clues that the distribution might be multimodal but it does clearly point to a long right tail and likely leptokurtosis. Indeed the grouped chart – unlike the ungrouped chart – looks fairly well behaved other than having nearly two thirds of the observations in the 1-2 group.
To stray off the main theme for a little, Gareth also had a good point in suggesting that we should be wary of using the statistical terms bias and outlier in the context of price relative distributions. One issue with using those statistical terms is that there isn't a robust probability model for describing the behaviour of price relatives which would allow bias and outliers to be well defined and identified.
Developing a robust probability model(s) will need to take into account such properties as kurtosis. I suspect that this would be a very specialised piece of research requiring input from academic statisticians. I also suspect that the most productive way forward may be to consider a probability model for prices first as these are directly affected by the pricing strategies of retailers etc. The model for price relatives could then be derived from the model for prices.