RPI/CPI User Group

last person joined: 5 months ago 

To foster co-operation and the exchange of information between the ONS, its advisory bodies and other users. Please see 'Aims of the RPI/CPI User Group' in the library for further details.

Topic: Skewness and geometric and arithmetic means.

1.  Skewness and geometric and arithmetic means.

Posted 17-04-2017 17:59
  |   view attached

Chris,

 

Rupert de Vincent-Humphreys' paper APCP-T(17)02 "RPI and CPI: a tale of two formulae" includes –

 

"The difference between the RPI (Carli formula) and CPI (Jevons formula) results reflects, in essence, the difference between the arithmetic and geometric mean of a sample, which in turn is proportional to the variance of the sample."

 

The variance of the sample does not tell the whole story.  The skewness – or asymmetry – of the sample is another important factor whose effects on how the different formulae behave need to be explored.

 

I attach a set of simple illustrative examples looking at the relative properties of the arithmetic and geometric means in left and right skewed and symmetric contexts.  The examples are artificial and were chosen for ease of calculation in Excel.

 

The first two charts below show that the same pattern of increasing variances can result in very different behaviour of the formulae depending on the direction the sample is skewed. 

 

The right skewed example shows what would be expected intuitively with the rate of change for the arithmetic mean being faster than that for the geometric mean with increasing variance.  In contrast, and perhaps counterintuitively, the left skewed example shows the reverse with increasing variance the rate of change of the geometric mean is much faster than the arithmetic mean– see charts below and attached spread sheet.

 

The symmetric example, which is not skewed and has larger variances than the skewed examples, shows a constant arithmetic mean as would be expected, but the rate of change for the geometric mean is approaching that of the left skewed example. 

 

Out of interest I also substituted the geometric mean for the arithmetic mean in the standard variance formula – I'm not sure if that is in any sense statistically meaningful . For these examples the geometric mean version is consistently marginally larger than the standard variance calculation.  There are, of course, no standard Excel functions for the new version but I think that the calculation is ok.

 

This is not at all straightforward, of course, but in the terminology of Rupert de Vincent-Humphreys' paper the geometric average may be less robust than the arithmetic average in circumstances of left skewed and symmetric distributions of price relatives.

 

The penultimate bullet point in APCP-T(17)03 "Re-addressing the formula effect" is potentially helpful here providing the answer to the question it poses is yes –

 

"Should we be trying to provide a better understanding of the divergences?"

 

Could ONS confirm that in the forthcoming paper they will be pursuing this question in its widest sense including the mathematics of averaging in addition to sampling, quality change and observational error issues.

 

All the best.

 

Arthur.





2.  RE: Skewness and geometric and arithmetic means.

Posted 18-04-2017 19:05

Arthur's calculation is correct.  However, in practice there is usually a large positive skew.  If we can approximate the distribution by a lognormal such that log X ~ N(m, s²) then the GM is exp(m) and the AM is exp(m+/2) so AM/GM is exp(/2).  The variance (var) is exp(m+)(exp(s²)-1) = AM²(exp(s²)-1) =AM²((AM/GM)²-1).  If AM = 1 for simplicity, then var = (1/GM²-1) so GM = 1/sqrt(v+1).


What Arthur's conclusion shows is that the GM is more robust to a large outlier to the right but the AM is more robust to a large outlier to the left.  This is easily shown by plotting the AM and GM of say (1, 1, 1, 1, 1, x) as a function of x.








3.  RE: Skewness and geometric and arithmetic means.

Posted 18-04-2017 22:42
Dear Arthur:
Your examples are very cleverly designed. My initial reaction was that you were showing how the arithmetic and geometric means react to outliers. Then I noticed that you make your low or high values account for four to six out of the 12 observations in the sample, so they cannot be considered extreme values or outliers.
Just the same I think your examples are skewed against the Jevons formula. I have reworked your examples so that values that exceed the original average 0.5 value are the counterparts of the values that fall below it, e.g. 0.4 in the second iteration of the left skewed example becomes 0.625 in the second iteration of the right skewed example, so that the geometric mean of the two values, 0.4 and 0.625 is 0.5. Note that this makes a big difference in the symmetric example, where the arithmetic mean shows higher and higher values with increasing variance but the geometric mean remains constant at 0.50.
In the variant of the symmetric example, this distribution would be considered skewed to the right since the upper values differ from the median value by much more than the lower values do, however if one  looks at the logs of the values there is symmetry. For example: ln(2.5)-ln(0.5)=ln(0.5)-ln(0.1)=ln(5).
I have added an Excel worksheet based on yours to the library. Sorry, there are no charts, just the data themselves.
Typically, price samples are skewed to the right, not to the left. They may contain high outliers that have not been removed, and if samples are small, it may not even be possible to remove them, especially if there is more than one in a sample. In such cases,  the Jevons formula is more reliable than the Carli formula. As IMF economists Silver and Heravi wrote: "The geometric mean is known to be more robust to outliers".
Best regards,
Andrew





4.  RE: Skewness and geometric and arithmetic means.

Posted 19-04-2017 09:20

Andrew's note raises an interesting point.  He quotes the claim that "The geometric mean is known to be more robust to outliers."  As I said yesterday, it is more robust than the AM to right-hand outliers but less robust to left-hand ones, which do also occur.  Attempts to clean the data by removing or correcting the outliers may concentrate too much on right-hand outliers.  When the GM is used, it is important to pay equal attention to left-hand ones to avoid a downward bias in the index.


Michael Baxter







5.  RE: Skewness and geometric and arithmetic means.

Posted 19-04-2017 13:54
Michael is correct that distributions of price relatives tend to have a right hand skew, though there are exceptions.  One of these is what are called "fashion goods".  These tend to be introduced at a high price and then fall in price until a new model or fashion is introduced at a high price. They are prevalent in the Clothing sector and also in electronic goods.  Such goods show price falls on any short term indicator, but in reality they are increasing in price in the longer term.  Diewert has recommended (about 2 years ago) to ONS that such goods be excluded from the RPI and CPI, but ONS has taken no action on this.

Having said this, the skewness of the distribution of price relatives is irrelevant to the choice of formula for elementary aggregates.  What matters is that the sample of prices collected is representative of the expenditures on that type of good (i.e. a PPS sample).  If this is achieved the arithmetic mean is an unbiased estimate of a weighted arithmetic mean, which is what we (most of us) really want.  The geometric mean is downwardly biased in such circumstances.  Whether the price collectors achieve this approximately is open for debate.  I believe they aim to do so.

If a so-called outlier is actually representing a skewed distribution, it is not really an outlier and should be treated as any other value.  To justify omitting values which seem extreme one needs to show that they are unrepresentative.

GJ


6.  RE: Skewness and geometric and arithmetic means.

Posted 20-04-2017 16:21
Dear Michael:
I don't dispute what you say. Neither, I am sure, would Mick Silver or his co-author. If you look at my paper "Common Sense Favours the Use of the Jevons Formula" I discuss the treatment of outliers on p. 9-12. Table 2 shows a simple example of a sample of four prices, where all prices are 100 except for one outlier value. For a low outlier, say 10, the Carli index is 77.5 and the Jevons index 56.234, so the Carli index is just 27% greater than the Jevons index. For the corresponding high outlier, 1000, the Carli index is 325 and the Jevons index is 177.828, so the Carli index is 82.8% greater than the Carli index. The differences get even more dramatic if one looks at more extreme low and high outliers. By assumption, the average price is assumed to be 100 so the closer the estimates are to 100 the better. To paraphrase Milton Friedman, the Jevons formula does better by low outliers than the Carli formula does by high outliers, even if it does worse by low outliers than the Carli formula. This is what I mean by saying it is more robust in the presence of outliers than the Carli formula.
Of course it would be better to identify and adjust for high and low outliers rather than falling back on the elementary aggregation formula to handle such problems, but this is not always possible. You may be interesting in knowing  that the quartile method was originally proposed for outlier detection in the Canadian industry product price indices in the 1980s without any transformation of the price relatives. It was discovered that this tended to ignore low outliers, which was why StatCan methodologists recommended much later in 2008 that the quartile method be applied with a logarithmic transformation of price relatives in outlier detection for the Canadian CPI. See the reference to the 2008 paper by Saad Rais in my paper. The Danish CPI uses the quartile method with a Hidiroglou-Berthelot transformation of price relatives. This is very similar to the log transformation, and serves the same purpose of making sure that low outliers are not ignored. Unfortunately, as far as I know, the quartile method has not been adopted in a systematic way in either the industry product price indices or the consumer price indices. The methods that were used when I worked there were decidedly ad hoc, and likely remain so.
Thank you, by the way, for your analysis of the variance of the different formulas, which was very well done.
Best regards,
Andrew 





7.  RE: Skewness and geometric and arithmetic means.

Posted 23-04-2017 15:18

There may be issues with Rupert de Vincent-Humphreys' paper but the analysis is grounded in actual price quote data which is to be welcomed.  The paper uses December 2014 data but for this post is restricted to December 2016 data in part because 2014 data could not easily be found on the ONS web site.

 

Rupert de Vincent-Humphreys' paper uses fashion data which is a particularly difficult area to interpret.  In contrast this post starts with the 2016 Q4 data for a more mundane product – baked beans – which should be better behaved.  Tins of baked beans are a staple non-seasonal foodstuff with limited quality change issues and so would hopefully be well behaved in terms of price quotes data. 

 

The following chart gives the frequency distribution for December 2016 –

 

 

The frequency distribution does not look like an iid sample drawn from either a normal or a log-normal distribution.  This is perhaps not surprising as the two theoretical distributions are continuous defined on the real numbers whereas the price relatives distribution is discrete and based on only a limited subset of the rational numbers.  This limited subset will be a product of, for example, the arithmetic of the small price range of tins of beans, the price collection methodology and the pricing strategies of retailers and manufacturers.

 

The range of values that baked beans price relatives take is from 0.81 to 1.58.  There is a question of whether these extreme values may be unrepresentative outliers and can be altered or removed from the sample.  This may be the case for the single 0.81 value but there are 5 price relatives at 1.58 and 32 at 1.5 - these latter two price relatives point to the possibility of the distribution of price relatives not being a single distribution but a mixture of two or more distributions.  The simplest such mixture of distributions is the bimodal normal but although that may represent a better approximation than the normal or log-normal in this case it could still be problematic. HHH

 

 

Rupert de Vincent-Humphreys' paper was, of course, not looking at a well behaved product but looking at fashion which is reputedly not at all well behaved.  A key chart from the paper is –

 

 

One question is whether the price quote data give any clues as to what may be causing this behaviour of the price indices post 2010.

 

The 2016 December price relatives for women's vests/strappy tops have a range from 0.199 to 10.179 and there is only one of each of these extreme observations though there are other observations that could be considered to be relatively close.  This range gives a factor of 5 in the left skewness and of 10 in the right skewness – note this is not a statistical measure of skewness.  The geometric mean is less robust to a large outlier to the left and the arithmetic mean is less robust to a large outlier to the right.  This 2016 data has apparent outliers in both directions which if this is typical of price behaviour for this product since 2010 could help provide an answer to the question implicit in the comment in Rupert de Vincent-Humphreys' chart above –

 

"Observed prices increased more than CPI; much less than RPI."

 

Unfortunately the very large range for 2016 December women's vests/strappy tops price relatives means displaying a full frequency chart is impractical – it would need to be 10 times wider than the above baked beans frequency chart.  The following women's vests/strappy tops chart uses the same vertical and horizontal scales used for the baked beans chart to allow some comparison across the different products –

 

 

 

The pattern again doesn't seem to resemble an iid sample from a normal or log-normal distribution.  There appear to be small clusters around increases of 10%, 20% and 33% which perhaps points to retailer pricing policies and may indicate a multimodal distribution. The second largest frequency is outside the chart area at a price relative of 2 with a count of 17 which further suggests a multimodal distribution related to retailer pricing policies. 

 

To conclude more research looking at the suitability of different averaging formulae for different individual products or services is needed.  Arguably this should not start with the most difficult products or services but look at those that would be expected to be relatively well behaved.  When an understanding of the better behaved products has been achieved then the issues with the more difficult products or services can be considered though those issues may primarily be with other parts of ONS price index methodology. 

 

My understanding is that historically the choice of different formulae for the RPI was derived from looking at the properties of the data but this was done a long time ago so is unlikely to reflect the current behaviour of the public or businesses that serve them.  I seem to remember that Andrew Lydon posted some time ago suggesting that such an exercise should be repeated.   There is an opportunity to do this in the development of the household index – an opportunity it is to be hoped ONS will take.

 

Also given the computing power that is currently available – which was not available to the designers of the RPI – ONS could consider more data related approximations than the normal or log-normal distributions.

 

All the best.

 

Arthur




8.  RE: Skewness and geometric and arithmetic means.

Posted 25-04-2017 10:17

Many thanks to ONS for providing the links to the historic data so quickly which allows more charts to be presented.

 

I have repeated the two December 2016 charts for 2014 – the year Rupert de Vincent-Humphreys' paper uses – and for 2005 which is prior to the change to fashion data collection and also before the financial crisis.

 

The story remains broadly the same in 2005 and 2014 as in 2016.  The two additional years add weight to the argument that price relative distributions may be multimodal unlike the normal or log-normal.  The multimodality of the baked bean price relatives moves around as would perhaps be expected depending on the rate and direction of price change. The women's vest/strappy top price relative distributions are less clear but again display clustering.  The 3 fashion charts appear to show an increasing range and right skew of price relatives over time though 3 months data is insufficient to draw any conclusions.  The women's vest/strappy top price relative distributions may indicate that the issues are more with the data collection methodology than the formulae.

 

Problems with using the normal and log-normal distributions as the basis for estimation were acknowledged in an ONS article from 2012 "Stochastic and Sampling Approaches to the Choice of Elementary Aggregate Formula" – https://www.ons.gov.uk/economy/inflationandpriceindices/methodologies/pricesuserguidancemethodologyanddevelopments

 

This article looked at the use of the Normal (arithmetic mean) and Log-normal (geometric mean) distributions as the basis for averaging price relatives and states –

 

"Statistical tests did not find evidence that either of the distributions fitted the data well. However, two-thirds of the elementary aggregates were closer to a log-normal than a normal distribution, indicating that for these a Jevons may be preferred."

 

The article did not consider multimodality and therefore the second sentence quoted above may simply reflect the asymmetry of the log-normal distribution and price relatives whereas the normal is symmetric.

 

It will be interesting to see if the forthcoming article will look at a wider range of approaches than the computationally simple normal and log-normal that were the subject of the 2012 article.

 

The 2014 charts –

 

 

 

All items Dec/Jan inflation was lower in 2014 than 2016 (CPI 1.2% compared with 2.3%, and RPI 1.9% compared with 3.2%).  The baked beans price relative again appears multimodal but in 2014 is left skewed.  There is also another cluster of 16 with a price relative of 0.61.  The women's vest/strappy top chart in 2014 is similar to that in 2016.  It is more compact than in 2016 with a narrower range that is closer to symmetrical with in relative terms – times 4.33 to the left and 5.33 to the right.  There remain apparent clusters at 1.25, for example.

 

The 2005 charts –

 

 

 

All items Dec/Jan inflation is similar for both CPI and RPI in 2005 at 2.6% and 2.8% respectively.  The 2005 baked bean chart again appears at least bimodal with the largest clusters not at a price relative of 1 but of 1.6 and 1.7. The women's vest/strappy top chart in 2014 is similar in shape to that in 2014 and 2016.  It is more compact with a narrower range than in 2014 that is even closer to symmetrical in relative terms – times 3 to the left and 3.45 to the right.  There remain apparent clusters at 1.15, for example, but with the sample size two thirds of those for 2014 and2016 it is more difficult to pick out patterns..

 

All the best.

 

Arthur

 

Links to price relative data

 

https://www.ons.gov.uk/economy/inflationandpriceindices/datasets/consumerpriceindicescpiandretailpricesindexrpiitemindicesandpricequotes  - latest data

 

http://webarchive.nationalarchives.gov.uk/20160105160709/http://www.ons.gov.uk/ons/guide-method/user-guidance/prices/cpi-and-rpi/cpi-and-rpi-item-indices-and-price-quotes/index.html  - recent historic data

 

http://webarchive.nationalarchives.gov.uk/20160105160709/http:/www.ons.gov.uk/ons/rel/cpi/consumer-price-indices/cpi-and-rpi-item-indices-and-price-quotes/rpt-cpi---rpi-item-indices---price-quotes.html - older historic data




9.  RE: Skewness and geometric and arithmetic means.

Posted 27-04-2017 18:23
I've been following this discussion with interest and Arthur's latest posting has prompted me to make a few comments.

From my own experience of examining price relative distributions, the most distinguishing feature I found was not skewness or multi-modality but extreme leptokurtosis. This is because, over relatively short time periods, the vast majority of prices do not change at all, producing a dominant spike in the distribution at 1. This is supported by almost all of Arthur's graphs (although the spike is strangely labelled as at 0.99, presumably because of the automatic scale generator). The one exception is for baked beans in 2005, where the spike is around 1.15 or thereabouts, presumably because of a general increase in bean prices over that year (although there is still a second, smaller spike at 1).

Any distributional approximations that don't allow for this leptokurtosis are likely to be misleading. I have seen this leptokurtosis remarked on in a few internal ONS papers, which are now presumably lost in the archives. It seems that ONS staff, like the rest of us, need to perpetually relearn the lessons of history. Perhaps we should require all price index statisticians to learn about the leptokurtic nature of price relatives.

We did try to approximate price relative distributions using a mixture of two Normal distributions, one tall and skinny, the other short and fat. This could replicate the general shape of these distributions but the approach was never completely satisfactory, not least because of the discrete nature of these distributions, as Arthur has noted. This arises because producers tend to increase prices in occasional steps rather than in daily infinitesimal increments. This is one reason why the Divisia index is not a good model for price indices.


10.  RE: Skewness and geometric and arithmetic means.

Posted 28-04-2017 08:07
Dear all

This has been an extremely interesting and useful discussion to follow and one that will certainly help me the next time I am involved in any discussions about the formula effect (including when it is raised in the Stakeholder Panel).

Michael's suggestion of plotting the AM and GM of (1,1,1,1,1,x) against x illustrates the  biases in Carli and Jevons nicely and is an approach I can see myself using the next time I am trying to explain the issue.

Many thanks to all those who have posted and I look forward to any future posts.


------------------------------
Jill
------------------------------



11.  RE: Skewness and geometric and arithmetic means.

Posted 28-04-2017 11:57
I think it would be best to avoid using the terms outlier and bias here.  We don't know the quantities represented by each price relative, so we cannot calculate a bias.

What is true is that the AM is more responsive than the GM to high price relatives and less responsive to low ones.  None of the values is necessarily an outlier.  The nature of the sample (PPS or otherwise) is very important here.

GJ


12.  RE: Skewness and geometric and arithmetic means.

Posted 01-05-2017 04:38
Yes, I think everyone liked Michael's example. As I said, it shows very well if you look at a range of high and low outliers, where, for example, for every high outlier 1000 there is a low outlier .001, the Jevons formula will seem more suitable for the high outliers, the Carli for the low outliers, but the Jevons will not show as grave a departure from one for low outliers, relatively speaking, as the Carli does for high outliers. This confirms the judgement of Mick Silver that the Jevons formula is more robust in the presence of outliers than the Carli formula or its other competitors. The CSWD formula is quite similar to the Jevons and they are identical for just two price relatives, but it compares poorly with the Jevons formula in its handling of outliers.






13.  RE: Skewness and geometric and arithmetic means.

Posted 01-05-2017 18:37

John,

 

Your point about the 2005 baked beans chart is interesting in a nerdish sort of way – sadly I ended up knowing too much about the price of beans because the Johnson Review at one point seemed to be about to use them as an example.  The beans for baked beans are imported from the US and Canada and will almost certainly be sold in US$.  The pound was strong against the dollar around 2005 and so the rise in price was probably due to a supply constraint – harvests fluctuate and with them prices.  There was another steeper increase in price at about the time of the financial crisis and that was probably mainly the result of a fall in the pound though if I remember rightly ONS changed the collection of baked beans about that time.  It will/might be interesting to see if the price of baked beans increases in 2017 with the fall in the pound following the Brexit referendum and what that does to the distribution of price relatives.

 

You also make a good point about leptokurtosis which adds another interesting dimension to the problem of the distribution of price relatives.  Its value is likely to be in exploring further approximations to the distribution of price relatives among continuous probability models.  Identifying continuous approximations tends to rely on grouping discrete observations.  This can hide the detailed structure of the price relative distribution as can be seen from the chart below for December 2016 women's vests/strappy tops which has been grouped unlike the first set of charts.

 

 

The distribution presented in this way shows few clues that the distribution might be multimodal but it does clearly point to a long right tail and likely leptokurtosis.  Indeed the grouped chart – unlike the ungrouped chart – looks fairly well behaved other than having nearly two thirds of the observations in the 1-2 group. 

 

To stray off the main theme for a little, Gareth also had a good point in suggesting that we should be wary of using the statistical terms bias and outlier in the context of price relative distributions.  One issue with using those statistical terms is that there isn't a robust probability model for describing the behaviour of price relatives which would allow bias and outliers to be well defined and identified.   

 

Developing a robust probability model(s) will need to take into account such properties as kurtosis.   I suspect that this would be a very specialised piece of research requiring input from academic statisticians.  I also suspect that the most productive way forward may be to consider a probability model for prices first as these are directly affected by the pricing strategies of retailers etc.  The model for price relatives could then be derived from the model for prices.

 

All the best.

 

Arthur.




'