RPICPI User Group

last person joined: 2 months ago 

To foster co-operation and the exchange of information between the ONS, its advisory bodies and other users. Please see 'Aims of the RPI/CPI User Group' in the library for further details.

Subject: The need for an uprating index

1.  The need for an uprating index

Posted 21-06-2013 15:52
  |   view attached
An issue I have been thinking about for a while is the distinction I believe exists between a consumer price index designed for uprating purposes and one for macroeconomic purposes. I attach a paper on the subject. This is intended as a think piece that will, I hope, stimulate debate, not as anything definitive. Comments are welcome!

2.  RE:The need for an uprating index

Posted 23-06-2013 10:42
Please see comments lodged in th library.


3.  RE:The need for an uprating index

Posted 24-06-2013 04:15

Thank you Gareth for your comments on Jill Leyland's upratings paper. 

I just wanted to say I was particularly intrigued by the idea of benchmarking consumer price series to annual benchmarks. This has, of course, long been practiced in constructing the GDP deflator, which Gareth recommends as the macroeconomic indicator of inflation. It is probably also used somewhat in producer price indexes where the revision period is long enough to permit. When I was working in the producer prices subdivision of Prices Division I persuaded the economist in charge of the farm input price index to use an annual benchmark price series for farmland prices. A quadratic minimization technique was used to interpolate a quarterly series.

The official CPI and RPI series are calculated without revision but there is no reason that this need be so for consumer price index series. Statistics Sweden's CPI is calculated as chain Walsh index with December links and has a 24-month revision policy. The US Bureau of Labor Statistics' chain CPI series is a chain Törnqvist index with subannual links and also has a 24-month revision policy. The long revision period has not prevented the Swedish CPI from being used as the Swedish Riksbank's target inflation indicator, or the US chain CPI from being recommended by both President Obama and the Republican House leader for major use as an upratings index.

Any switch to a similar chain formula (my preference would be a chain Edgeworth index with annual average links) would automatically open up possibilities for benchmarking that don't exist right now. It is both feasible and desirable.

A revision policy of 24 months or longer would also open up other possibilities that don't exist now. The Rothwell formula, which does not require revision, has been the most commonly used monthly-basket formula for seasonal goods in consumer price series. It was what the ONS used for fresh fruits and vegetables in the CPI until recently. In my opinion, the current treatment of holiday trips and horse race admissions in the RPI is nothing but applying a very crude seasonal adjustment procedure to a Rothwell index. The Balk formula, which can accommodate moving seasonality (e.g. seasonal purchases associated with Muslim holy days) or irregular seasonality (e.g. seasonal purchases associated with Easter), could likely be employed in consumer price series where needed if there were a revision period of at least 24 months.

4.  RE:The need for an uprating index

Posted 26-06-2013 16:18
I was deliberately being cautious about advocating revisions.

I think the public sector might cope with difficulty (and cost).  However it might throw the private sector into chaos through the loss of certainty.


5.  RE:The need for an uprating index

Posted 27-06-2013 18:27
I have posted some comments that are in the library.

6.  RE:The need for an uprating index

Posted 28-06-2013 12:47
It shows how useful Jill's document is that it has already provoked so much comment.

I read Anne Harrison's interesting comments and I would just like to reply regarding two issues she raised.

First, regarding insurance expense, the net premiums-gross expenditure approach is used to calculate the UK CPI or UK HICP. It is not used to calculate the RPI, where the gross premiums-net expenditure approach is used. The gross premiums-net expenditures approach would seem to be more appropriate to an upratings index and is certainly more appropriate to a cost-of-living index, which is why it is used in the RPI. The net premiums-gross expenditure approach used in the HICP would seem to be more appropriate for that purpose although the justification for using the gross premium rates to determine prices for an index with a net premiums weight would seem to be a little thin. Ralph Turvey called it a practical approach. Others might dismiss it as makeshift. The problem is that it is difficult to reconcile the requirements for observed prices and expenditures and positive price indexes and expenditure weights. If one calculated the insurance index as the weighted average of a gross premiums index with a gross premiums weight and an insurance claims index with a repairs weight, then the net premiums-gross expenditures approach would be equivalent to the gross premiums-net expenditures approach. However, the net premiums index might very well go negative if insurance claims exceeded gross premiums, which would be disturbing for many index users.

There would certainly seem to be a case for replacing actual claims with expected claims in the expediture weight for insurance, mirroring methodologies that have been introduced in the national accounts of several countries. If actual claims exceeded expected claims over the recent period this would reduce the relative importance of premium rates to repairs costs in the combined weight for insurance premiums and repairs.

In a situation that might occur where gross premiums were rising because of increased risk while repair rates were stable due to slack in the economy, the existing treatment of insurance by the HICP would seem to give an underestimate of the increase in insurance costs and repair expenditures combined.

Pricing of claims under the net premiums-gross expenditures approach could be problematic in some cases. For example, for personal property insurance claims, jewellery might simply be replaced by the insurance company in kind and the household concerned wouldn't know the monetary value of the replacement. This does not constitute a problem under the gross premiums-net expenditures approach since claims are simply ignored using this approach as far as pricining is concerned.

I don't think that Ms. Harrison's remarks about credit card payments and such really constitute a valid objection to the payments approach. Still less do they constitute a valid objection to the lessor's deductble cost approach used for measuing owner-occupied housing costs in the RPI and in the Canadian CPI. Any modelling of credit transactions in the CPI would look exclusively at mortgages and similar installment payment arrangements, for real estate, motor vehicles and other consumer durables. Credit card payments would be excluded.


7.  RE:The need for an uprating index

Posted 28-06-2013 16:48

I would like to express support for Jill's proposition that there are legitimate reasons for using a different price index for macro-economic general inflation purposes and for what Jill calls uprating. For the first purpose I believe the CPI (and perhaps the new CPIH) is a good index. For the latter purpose the objective is to say how much do wages or benefit payments need to be increased to maintain the collective recipients' ability to purchase goods and services.

For wage purposes RPI(J) is probably as good an index as any. For pensions it might be the pensioner version of RPI (J).

Jill explains many of the reasons why the weights and in some cases the items might be different in an uprating index. I agree with most of them.

Of course the elected government is free to make the decision that benefits (or controlled prices such as those for utilities) will only be changed in line with the CPI, which makes sense from the perspective of the macro public sector finances. However when looking at the purchasing power of these benefits it is the uprating index which should be used.

8.  RE:The need for an uprating index

Posted 29-06-2013 15:50
It is not clear to me why people are assuming that wages, pensions and benefits should be uprated in such a way as to maintain their purchasing power in absolute terms.

I would have thought that they should be uprated in such a way as to maintain their relative purchasing power compared with the national average.  This would imply use of an index of average earnings rather than a price index.  

Wages will tend to follow average earnings automatically.  If receipients of pensions and benefits are only linked to prices their incomes will not reflect economic growth and they will fall progressively further and further behind the working population.

Ultimately this becomes unsustainable.  But why let it get to that position in the first place?


9.  RE:The need for an uprating index

Posted 01-07-2013 10:51
Back in the mists of time, civil service pensions were not indexed.  HMG agreed that this should change and proposed to uprate them in line with average earnings.  The unions were outraged - it was a time of pay freezes and high inflation - and won their battle to have the uprating changed to the RPI.  I remember this every month when my civil service pension arrives!


10.  RE:The need for an uprating index

Posted 01-07-2013 11:44

I suspect that Anne's and Gareth's contributions both illustrate the folly of making long term decisions in times of more than usual economic upheaval and uncertainty.  For example, if the current low growth continues for many years then indexation by the CPI is likely to appear advantageous if unsustainable.


I think though that the argument for an uprating index based on prices rather than incomes is an argument about filling a gap in the indicators.  That argument is arguably different from the political argument of which index should be used for which purposes.   Government can choose to use which index it thinks appropriate in the prevailing economic and political circumstances of the time.


An uprating index based on prices will provide another perspective on what is happening in the economy and help inform the debate on such issues as indexation - well that is my hope anyway.


11.  RE:The need for an uprating index

Posted 02-07-2013 11:46

I entirely agree with Arthur's second paragraph - only as usual he has put it more elegantly than I could - that the prime need is to fill the gap in the indicators. What actually determines how salaries, pensions etc are uprated is a political, market-led, negotiated or contractual (but not statistical) decision depending on circumstances. But you still need an "uprating" prices index as a gauge or as part of negotiation.

In the last company I worked for, as the company's Economic Adviser I was asked every year by the head of HR for the growth in consumer prices over the past year and a forecast for the next year for all the countries in which the company operated. This fed into salary reviews. It did not determine salary rises on its own since these also depended on what the company could afford, the individual's performance and any change in their responsibilities along with information on comparable salaries elsewehere but it was part of the mix. For the UK it was difficult as neither CPI nor RPI seemed ideal for the purpose.

12.  RE:The need for an uprating index

Posted 01-08-2013 12:16

Many - long overdue - thanks to all those who have commented on my note. Starting this sort of debate is precisely what I hoped to do.  In due course I will update the paper.

I would like now to pick up some of the points that have been made - and in doing so also raise one or two more questions. I must first apologise for waiting so long to do this and hope that readers will not have forgotten the argument.  My original paper, along with replies from Gareth and Anne, is in the library.

1. First, an important point made to me privately is that to be comprehensive the paper would need to look at the desirable characteristics of a macroeconomic index. I accept this - any thoughts on this would be welcome. I have implicitly assumed that the HICP/CPI sets the standard for a macroeconomic index - apart, of course, from its lack of owner occupied housing. But is this right?

2. Payment vs Acquisition vs Use. My paper argued that an uprating index should normally be based on payment as this is the point at which the household budget is affected. Anne stated that payment was usually rejected as a) it is theoretically inferior to the other two concepts and b) practically impossible to implement. I am puzzled as to why payment is considered theoretically inferior?  I accept Anne's point that payments by credit card would need a little thinking about (and would depend partly on the treatment of interest payments) but the practical problems seem to me minor compared with those associated with use (what "use" does one get from food thrown away because it was not eaten by its sell-by date or from the article of clothing that languishes at the back of the wardrobe because you decide after buying it that it really does not suit but you were unable to return it?). Indeed I would argue that "use" is easily a more unsatisfactory concept, both theoretically and in practice, than either acquisition or payment - I agree with Don Sellwood's arguments on this (set out under "A cpi for uprating").

3. On interest, I don't think there can be much of an argument about the service charge being included and Anne makes some useful comments here.  As to whether the rest of the interest payment should be included - one can indeed argue that it is a means of financing consumption rather than consumption or one can argue that it satisfies the consumer's need to have something now rather than later. What does everyone think? I would argue that borrowing to buy is now part of life.

I don't think, though, the fact that households receive interest as well as pay it alters the argument. This is not national accounts where everything has to balance. For price indices we are only looking at one side of the equation: what households spend.

4. The issue of aggregation formulas has been touched on. My (strong) feeling is that we should use the same formulae for both sorts of index unless there is a very clear reason not to. The confusion and arguments arising from the formula effect at lower level aggregation have been serious and damaging.  I can see Gareth's arguments that if there are elements in an uprating index that are erratic (such as mortgage interest payments) there could be a case for smoothing them. But this, I think, would be acceptable since it would be easily explicable to a lay person. (As a more general point I think it is desirable that as much as possible of the way a price index is compiled should be both explicable and justifiable to a lay person. It is surprising how much can be explained if you do it carefully.)

5. Housing. More than one person has suggested that I should have thought about final sales or downsizing. And not everyone is entirely happy about including something that has an investment element. So let me talk about housing - specifically owner occupied housing - a little more.

The more I have thought about this, the more I believe that the argument that because housing is part investment house prices, or variables related to them, should not be included does not hold water. I do though think that the weight accorded to house prices should be adjusted downwards to allow for certain elements associated with its investment characteristics (see below).

First, why should it be included?

a. Shelter is a necessity. One can rent or buy and an index that reflects costs to the household budget should be neutral between the two.

b. The fact that owner occupied housing is treated as investment in national accounts is irrelevant in my view. We are not dealing with national accounts but with a price index. Further, national accounting (within its well known limitations) is a comprehensive system; whether something is included as investment or consumption does not a priori alter GDP (second order effects apart). But if we exclude house purchase from our price index we are missing something of crucial importance to most household budgets.

c. Anne argued, "The whole of the present economic crisis has its origins in assumptions that house prices would always increase and so I would have grave misgivings about building this assumption into such a sensitive are as consumer price indices."  I assume you did not mean to say "the whole" as distinct from "part of", Anne, but clearly housing was a significant factor. And a price index would not include an assumption but what has actually happened to prices. But I think the link between house prices and the crisis makes it more important to include them (in this case also in a macroeconomic index but I will leave that argument to others.) House prices have risen in this country primarily as we have built too few in the last 2-3 decades to match the growth in demand; speculation on the back of this has added additional froth. But the result is that it costs more for people to buy a house (and indirectly also more to rent) and this should be reflected in an uprating price index.

I could make other points but I think the substance of them was in my original note so will leave it there for now.

Let's look at the qualifications.  I don't think final sales come into it. Again, in a price index we are looking only at costs - not at income or monies that accrue.  Pure investment buying eg for "buy to let" is another matter. The weight accorded to housing should, in my view, be adjusted to reflect the proportion of purchases that are done for this reason.

Downsizing. Thinking about this made me realise I had made one mistake in my original note. I had argued that what I called the "housing ladder" element of house purchase - the profit a houseowner has made on his existing house which is then applied to a subsequent house purchase - should be excluded. I still believe that but in addition I think the initial capital the home owner invests, plus the repayments made under any mortgage, should also be excluded from any subsequent house purchase since they should only be counted once. When we come to downsizing then, one would not normally include those purchases at all other than if there are still ongoing mortgage costs. The consequence of this would be to reduce the weight given to house prices in an uprating index. All of this of course does require not just careful thinking but also a lot of knowledge of the housing market. I assume though that the knowledge to be found in entities such as Halifax, Nationwide, the House builders' federation and similar entities would give one a reasonable steer.

13.  RE:The need for an uprating index

Posted 02-08-2013 18:59
I don't think I would go so far as advocating different elementary formulae for different purposes.  Rather I would argue that the Carli formula is superior in an uprating index (which is probably the major concern of most people) and should then also be used in the GDP deflator (which I would argue is the macroeconomic inflation index).  This is what was done in the UK until a couple of years ago.

Regarding smoothing, my argument was actually that we should "de-smooth" by placing the emphasis on long term bias rather than variance.  The one exception I raised was OOH costs where changes in interest rates can cause huge swings in the inflation measure.  I don't think of this as variance in a statistical sense, but a fundamental problem with the methodology.  In the 1970s the RPI measure of inflation averaged 14% per year.  However there were two peaks of 27% and 25% respecively in between which was a trough of 8%.  Much of this variation was due to changes in interest rates.


14.  RE:The need for an uprating index

Posted 03-08-2013 16:34
  |   view attached



My view on using the "same aggregation formulae" for indices differs from yours and possibly Gareth's.


I am reluctant to support statistics that potentially hide uncertainty and give the impression that statistical methodology is better than it is. 


The formulae conventionally used all have their problems and therefore provide one of the dimensions of the uncertainty associated with price indices.


I am a little wary to offer simple illustrative examples as they can be over interpreted but hopefully the attached spreadsheet might be useful.


The examples are simplistic and far removed from the real world but hopefully illustrate a couple of points. 


What the examples do not do, and are not intended to do, is to suggest Carli is superior to Jevons or vice versa.  The intention is just to offer another illustration from a layman's perspective that both produce questionable results in different circumstances.


Basically the examples I have chosen show what can happen to arithmetic and geometric means of price relatives with symmetric additive changes in prices.  I would argue that a layman would reasonably consider that these symmetric changes in price to cancel each other out and that "inflation" is unchanged because the aggregate cost of the two items remains the same in both time periods.


The simplest example is the top left hand one.


In this case there are 2 items A and B each with a price of £1 at time T1.  The price of A decreases by 50p and B increases by 50p at time T2.  Therefore the cost of the two items remains at £2 in total at time T2 the same as at T1.


The arithmetic mean of the price relatives is 1 which indicates no price change whereas the geometric mean is 0.87 indicating a fall in price which is arguably counter intuitive.


The other two examples on the left hand side vary the price of one of the items at T1 to £2.  These examples show that the two averages can both imply consistent increases or decreases in "inflation". Again I would argue that the layman would again expect no change in "inflation" with these examples.


The first 3 examples illustrate how both means can give undesirable results.  (As an aside the numbers can easily be changed so that the geometric mean returns a mean price relative of 1 - change the initial value of B to 50p in the third example.)


Now it can be argued that the price changes of 50p are untypically large as they equate to things like sale prices or two for one offers.  The calculations to the right show what happens when the price change is reduced.  The patterns remain the same but differences reduce.  By the time the price change reduces to 1p there is very little difference in practice between all the calculations of averages.


These additional calculations illustrate that although there may be theoretical problems with both methodologies they may not always represent problems in practice depending upon the data.  (As a further aside not all possible simple aggregation formulae will be produce undesirable results with these examples but such formulae are likely to demonstrate problems in other circumstances.)


And finally this brings me to a more general point.  One of my concerns with price index methodology is the mathematics and the way it is used.   With my use of simple examples I am potentially guilty of generalising from the particular but I hope that I have made it clear that that was not my intention.  In general it seems to me that proponents of different methodologies are prone to selecting particular isolated mathematical results to support their case.  In their defence it may be because there is no comprehensive body of supporting mathematics to describe how different functions will behave in different circumstances, or the mathematics exists but in practice is too difficult to access.


This is not to imply that mathematics has the answers which it clearly does not because the judgement of various disciplines is likely to dominate in describing such complex phenomena.  However those judgements need to be based on a sound mathematical foundation.




15.  RE:The need for an uprating index

Posted 04-08-2013 02:31


I can't find your file attachment, but your first example is clear enough with it. I take your point. The expenditure would be the same after A had fallen in price and B had increased in price so shouldn't we favour the formula that shows no change in price (Carli) over the one that shows a decrease (Jevons) If that was all there was to it, I would agree with you. But, as we both know, the RPI is a chain index with links every January. So the test that really matters is how well do the two formulas behave when they are chained. The answer is that the Jevons formula behaves very well and the Carli formula doesn't behave well at all.

Below I have shown what happens if the prices return to their level in period 0 in period 2. The link relative for the Jevons index for period 2 is just the reciprocal of its link relative for period 1, which you find unreasonable, so the chain index for period 2 with a period 0 base equals 1.0, as one would expect since the prices of A and B are the same in both periods.

The link relative for the Carli index for period 2 is 1.333. The situation going from period 1 to period 2 (identical to period 0) is comparable to the one going from period 0 to period 1 in the sense that the buyer can buy a unit of A and a unit of B for £2 in both periods. But whereas going from period 0 to period 1 the Carli index showed the nice link value of 1.0, so intuitively plausible, going from period 1 to period 2 it takes a value of 1.333. The difference is easily explained. Implicitly, the Carli links are what you would expect a Laspeyres price index to be if the quantities were inversely related to the prices. In the case of the period 0 to period 1 comparison the two items have the same price so they are given the same weight. In the case of the period 1 to period 2 comparison, item A's price is only a third that of item B's, so it is weighted much more heavily by the Carli index than B, leading  to the big increase in the Carli link.

So although prices are the same in period 2 as in period 0, the chain Carli index shows that prices have risen by a third. This is why one national statistical agency after another has moved away from the Carli formula.

Possibly in certain special circumstances, its use could be defended. Arguably, it might be a suitable formula if the Carli index is the byproduct of pps samplng based on expenditures, and a new sample is drawn every year or every time the elementary aggregates are linked. You have convinced me that perhaps the total ban on the Carli formula by Eurostat is excessive. But is there anyplace in the existing RPI or CPI where the situation that I have just described holds. I rather doubt that there is.

0 1 2
A 1 0.5 1
B 1 1.5 1
Link Relatives
0 1 2
A 1.000 0.500 2.000
B 1.000 1.500 0.667
Carli 1.000 1.000 1.333
Jevons 1.000 0.866 1.155
Chain Index
Carli 1.000 1.000 1.333
Jevons 1.000 0.866 1.000

16.  RE:The need for an uprating index

Posted 04-08-2013 07:38
Arthur, Andrew, Gareth

The simple point I wanted to make was that whatever aggregation formulae are used, they should be the same in both an uprating and a macroeconomic index. Unless there is a very good reason to the contrary. While we may have different views on which lower-level aggregation formulae should be used, I don't think any of you have suggested an example where they should differ between the two types of index? Anne, in contrast, did suggest in her note that upper level aggregation might differ (although I don't think she had strong views) and my post of a few days ago was intended to respond to that.  

Could I suggest that to avoid confusion this particular thread is limited to whether and where there should be differences between an uprating and a macroeconomic index?

On smoothing, I agree with the points you make  - ie that in an uprating index long term accuracy should generally be prioritised over higher variance but that there might be a case for smoothing anything which causes really substantial year to year swings in the overall index as mortgage interest has done in the past. 

17.  RE:The need for an uprating index

Posted 07-08-2013 14:10
I think there are two possible reasons why different formulae for elementary aggregates might need to be used for different uses or purposes:

a) a difference in the target index which one is trying to approximate (e.g. Fisher vs Laspeyres)

b) a difference in the loss function used to evaluate such approximations.  Mean squared error has conventionally been used and is suitable for short term considerations.  On the other hand, long term considerations might indicate use of the bias as loss function.

One of the reasons we are not making much progress on settling the question of elementary formula is because these two issues have not been decided.  We are trying to solve an ill-defined problem.

If we take annual chain linking as a given, I would suggest that for uprating purposes, the target index should be Laspeyres (i.e. fixed basket for a year) and the loss function should be the bias.  ONS' research so far suggests that both of these choices tend one to prefer the Carli formula.