RPICPI User Group

last person joined: one month ago 

To foster co-operation and the exchange of information between the ONS, its advisory bodies and other users. Please see 'Aims of the RPI/CPI User Group' in the library for further details.

Chain linking – ONS, the Technical Panel and CPI and CPIH

  • 1.  Chain linking – ONS, the Technical Panel and CPI and CPIH

    Posted 22-01-2018 15:21
      |   view attached

    Through 2017 I was engaged in a rather one sided and unproductive conversation with the Technical Panel about chain linking algorithms. The panel secretariat did, however, share an unpublished paper by Professor Bert Balk which proved very interesting.  The paper indicated that the HICP is also subject to problems of chain drift (price bounce) similarly to the RPI.  CPI uses HICP methodology but unlike that index and the RPI it is not linked once a year but twice.  This would imply that potentially chain linking may be more of a problem for CPI and CPIH.


    Bear with me while I recount the conversation with the Technical Panel about chain linking algorithms which serendipitously revealed Professor Balk's paper and the potential issue with the CPI.  In two short notes to the panel for the May and September meetings I raised two questions –


    First note question –"Is there a theoretical basis which implies that the current chain linking algorithm is the only one that can be used?"
    Second note question – "What is the likely scale of the effect of using the current chain linking algorithm (CCLA) on existing UK price indices?"


    The answer I received in the minutes to the September meeting was –


    "Prof. Balk's draft paper on mixed-form indices should address this question somewhat and has been shared with Mr. Barnett. The panel thought this was sufficient to answer the question posed."


    Professor Balk's draft paper was not published alongside the minutes of the September meeting – he has since agreed to his paper being placed on SUN, see note below – which made it difficult for UG members and others to assess the statement in the minutes.  However the language – "somewhat" and "sufficient" – does not suggest much precision of thinking has gone into answering the two questions.  My reading of Professor Balk's paper is that it does not directly address either question.   I submitted a further note to ONS asking the panel to expand on the minutes but this note was not accepted for inclusion on the agenda by the panel chair.


    The paper is particularly interesting as it looks at chain linking where it actually occurs in the calculation of an index – in this case HICP.  Chain linking is not carried out at the level of the elementary aggregates with the issues that raises in the use of Carli, Jevons or Dutot but at the level that weighting data is available. 


    Professor Balk describes the issue with the HICP as "… there seems to be a whiff of double-counting here."  The solution offered is related to the use of logarithmic means instead of arithmetic means. Such an approach seems counterintuitive as the problem is arguably the result of the chain linking algorithm not the various averaging processes used elsewhere in the index calculations.  However Professor Balk's solution offers another potentially simpler way of calculating the effect of using the current algorithm on UK price indices than changing the chain linking algorithm. 


    It will be interesting to see if ONS or the Technical Panel respond to the challenge implied by Professor Balk's paper to the UK CPI and CPIH with their double chain linking.


    All the best.




    Note:  Professor Balk's paper "Mixed-Form Indices: A Study of Their Properties" has not yet been published.   It has been presented at the last Ottawa Group meeting, May 2017, and will be available at the OG website (www.ottawagroup.org) when the site has been updated.


    At this stage Professor Balk would welcome any comments.


    Mixed-Form Indices.pdf   235K 1 version

  • 2.  RE: Chain linking – ONS, the Technical Panel and CPI and CPIH

    Posted 22-01-2018 16:49

    You are correct that price bounce (or more generally chain drift) affect CPI and CPIH and not just the RPI.

    However, I have shown numerically  (and posted on this website more than once) that the difference in chain drift between RPI and CPI at the All Items level is very small and well within the rounding error of published figures.

    Chain linking is an undesirable necessity.  It conceals underestimation of inflation between chain links by eliminating the change in average prices between baskets of successive years.  This is particularly notable in respect of Clothing inflation and Rental inflation,  It also tends to support the "no revisions" policy for leading inflation measures.

    The best thing we could do about it is to do it less often (perhaps every 5 years as used to be the case with the National Accounts - then known as re-basing).

    I don't think that changes to the base month will have much effect.

    Changes to the chain -link formula might have more effect, but we have no evidence to judge at present.

    The "no revisions" policy also needs reconsideration.  I understand that the government and others do not want to alter contracts which are linked to inflation.  However such contracts could be based on the first publication of an index only,  with corrections coming through later but without contractual force.


  • 3.  RE: Chain linking – ONS, the Technical Panel and CPI and CPIH

    Posted 23-01-2018 17:10

    Dear Arthur:

    Thank you very much for this paper. Bert is absolutely brilliant and one of the leading advocates of a seasonal-basket approach to seasonal goods in price indices. Anything he writes deserves a careful read. Just glancing at it I can see that he reprises the argument of his 2006 paper "Measuring and Decomposing Rates of Inflation Derived from Annually Chained Lowe Indices". Although in this paper he references Martin Ribe of Statistics Sweden as the source for formula (14), it can be easily recognized as the decomposition prescribed by the ONS for the Retail Prices Index. Bert's source is an unpublished 1999 mimeo by Mr. Ribe, but Richard Campbell informed me that this method is described in The Retail Prices Index Manual, published in February 1998, and was used in analyzing the RPI for some time before that.

    With regards to double counting, if one has a price index that doubles every year,  the inflation rate will be 100% in year 1 and 100% in year 2. Inflation from year 1 to year 3 will be 300%. Is there double counting in attributing a third of the inflation to year 1 and two thirds of it to year two? In a sense, there is, because the inflation rate is the same in both periods. Just the same, it would certainly be true that if an item cost £1 in year 1, £2 in year 2 and £4 in year 3, two thirds of the difference in price between year 1 and year 3 prices stems from the difference in prices between year 2 and year 3.

    Starting from December 2007 Statistics Canada began publishing a list of the major upward and major downward contributors to the annual CPI inflation rate. At that time, there was a change in basket only every four years, so the Canadian CPI annual inflation rate was only impacted by basket shifts slightly less than a quarter of the time. The official position of StatCan at the time, as expressed in the 1995 CPI reference paper, was that it was not possible to devise an arithmetic decomposition of inflation rates with changing baskets. Instead, contributions to change derived from unlinked series based on the new basket were used as proxies. Prior to the switch from a 2005 to a 2009 basket I recommended that StatCan switch to estimating contributions to change based on the ONS method for the months May 2011 to March 2012 affected by the switch. I also proposed that Bert's method be used to calculate alternative contributions to change, which could be published if they differed substantially from those generated using the ONS method. (For single-digit inflation rates, the differences would never be substantial.)

    My proposition was rejected, and the tables of major contributors were compiled based on contributions for the unlinked 2009-based indices, which did differ substantially from those generated by the ONS estimates. Starting with the February 2013 update of the Canadian CPI, StatCan did switch to the ONS method for calculating contributions to change as the basis for calculating its table of major upward and downward contributors where there is a basket change. (The February 2013 update also marked a switch to biennial basket updates, so there was more frequent use of the ONS method.) However there was never any acknowledgement that things were done differently previously, let alone the publication of corrected tables for major contributors for December 2007 to March 2008 and May 2011 to March 2012 based on the ONS method. Nor, regrettably, was any effort made to calculate contributions to percent change using  Bert's method.

    ONS could still choose to do what StatCan has failed to do: produce annual contributions to change using Bert's method, to be noted and published where they differ substantially from those generated by the ONS method. Here "substantially" means that the published contribution would be different when rounded to the second decimal place, as is the current practice in the CPI detailed briefing note.

    Thank you again for bringing the Group's attention to Bert's note.

    Best regards,


  • 4.  RE: Chain linking – ONS, the Technical Panel and CPI and CPIH

    Posted 17-02-2018 17:12

    I have received a very helpful short email from Chris Payne who is a member of the Technical Panel's secretariat.  The email provides more information about the Panel's deliberations on chain linking – the substantive paragraph from the email is at the end of the post.


    The extracts from the email highlighted below in italics would suggest that there is a strong argument for revising the relevant section of the minutes to the September meeting of the Technical Panel to reflect more precisely the Panel's thinking. The Panel minutes for September were revised in February 2018 so there is a precedent for the Panel Chair to make such a revision.


    The email states that the Panel –


     "… agreed that there was no theoretical reason that the current chain linking algorithm is the only approach". 


    This implies agreement that it is possible to modify the chain linking algorithm to provide the desirable properties from a range of elementary aggregate formulae including all three that are currently used in the UK – Carli, Jevons and Dutot. 


    Professor Balk's paper shows that problems with chain linking occur at a different level in an index calculation than the elementary aggregates and again the chain linking algorithm could be modified to provide the overall index with the desired properties without the need to change the averaging functions.


    However the current chain linking algorithm does have advantages.   It is simple and fits easily into the no revisions convention.  The existence of a theoretical anomaly does not of itself require a change to the chain linking algorithm unless it would make a practical difference to the calculation of an index.  And indeed the email goes on to state that –


    "None of the panel thought that this was a fundamental issue with our price indices…"


    ONS price indices include the RPI with its reliance on Carli.  Therefore one interpretation of this statement is that the Panel's view is that the current chain linking algorithm would be likely to represent sufficiently good an approximation for use with the RPI and there is no fundamentally different issue with chain drift for that index. 


    However, even in the absence in practice of any significant chain drift there remains the need to identify the other sources of variation that drive the formula effect between the RPI and CPI. 


    Work on the formula effect is particularly important and in need of being given higher priority by ONS as it is a key factor in the choice of elementary aggregate formulae in a household index like the HCI.


    All the best.





    Extract from the Technical Panel Secretariat email, Thursday 8th February 2018
    The September APCP-T minutes covered the points from the AOB discussion, so I'm afraid I can't add much in the way of a fuller explanation. To clarify, they agreed that there was no theoretical reason that the current chain linking algorithm is the only approach, and they suggested that Bert Balk's paper, which we have shared with you, should answer your questions. None of the panel thought that this was a fundamental issue with our price indices, and so Nick* has decided not to pursue the matter further with the panels.
    *Nick Vaughan Chair of Technical Panel.