In the margins of the 22nd February 2018 Public meeting "Measuring household inflation in the UK: Introducing the Household Cost Indices" I had a very brief discussion about an analysis ONS is carrying out into the effect of quality adjustment or non-comparable substitutions at the overall index level. This is in contrast to the analysis the Johnson Report carried out at item level though if my interpretation is correct the methodology ONS appear to be using is different from that used by Johnson which may reduce the value of the analysis.
The discussion was brief and if I understood it correctly the latest ONS methodology appears to be comparing a newly calculated direct index with the published chain linked index. Such a methodology may only adequately allow for the between year implicit quality adjustment and fail to allow for the effects of within year implicit quality adjustment.
To illustrate the potential issues I am referring back to a 2015 library entry that contained an analysis of within and between year quality change derived from the Johnson Report. http://www.statsusernet.org.uk/viewdocument/within-year-and-between-year-implie?CommunityKey=3fb113ec-7c7f-424c-aad9-ae72f0a40f65&tab=librarydocuments
The analysis presented in the library entry contains the following short table analysing quality change for the selected products in the Johnson Report plus rental equivalence (RE).
The table shows that between year quality change is for the most part larger than the within year change. The exception is T-shirts where the within year change is the largest of any change by a very large margin. This behaviour of quality change in T-shirts suggests a possible more general issue with clothing at least which points to the potential importance and scale of within year quality change at the overall index level.
Average yearly change
The RE index has the only negative within year quality change but given the date of the analysis in May 2015 the calculations may have been carried out on the problematic series that resulted in the de-designation of CPIH which may have affected the numbers.
Methodologically the Johnson Report analysis of quality change compared Dutot elementary aggregate indices with an average price index. There is a logic to the Johnson approach as the average price index excludes all quality adjustment/non-comparable substitutions. Johnson argued that Dutot is also effectively the ratio of average prices and so conceptually fits with a comparison with an average price index.
Building an overall index based on Dutot alone to compare to an overall average price index may represent a considerable challenge as Dutot indices may not be available for all products and services. However if it is practical, in part at least, it may have advantages. It may, for example, enable a separation of the analysis of the quality change effect from the formula effect between Jevons and Carli. If my interpretation is correct the new ONS analysis for the overall index seems to be intending to compare the published index with a direct index presumably using all the same elementary aggregate formulae as the published index. This may partially confound these two important effects.
All the best.
Note: as the old library entry makes clear the terminology in this area is potentially confusing. The above and much else that is written refer to quality change, implicit quality change and non-comparable substitutions all to mean the latter. There are explicit quality change methodologies like hedonic regression but the most used methodology is non-comparable substitutions.
Just to clarify, the review of quality adjustment procedures that we're currently undertaking is intended to look at a range of indicators that we could potentially use as a dashboard to monitor quality adjustment methods and whether they are working as we'd expect. One of the indicators we are looking at is an 'Implicit Quality Index', which is broadly what you have described below. This is calculated at the item level not at the aggregate level. This should help to show the impact of quality adjustment on items within the basket.
This research is still at an early stage. We will also be looking at other indicators and are open to suggestions and feedback. We will of course aim to present our work at a future APCP for further advice and feedback.