Richard Davies

Economist

Data

UK prices

By

economics, business, big data, firms, wages, finance

This page houses my latest work on prices and inflation, using 'micro data' to investigate inflationary puzzles and implications. The UK is in a unique position here, with CPI micro data released with a lag of only one month, allowing work on prices, inflation and real wages to be conducted in (close to) real time. I have used this to create a long-run prices database (LRPD), which I update monthly, and make available here. Select an item using the drop-down box below to begin investigating the data.

Papers and ongoing research

Some papers and blogs using this data include:

This is ongoing research, if you have any comments or would like to use the data, please get in touch.

Long run prices database

I update the LRPD monthly, the latest update was 17th February 2021. The data is stored in the following five files (all csv):

/// Prices. (Full database). This contains the price data. It is a very large file (there are currently ~41m observations.) To conserve memorty all supplementary details are kept in other databases that can be linked to when needed. Please do not open this in MS Excel, as this will result in the loss of most of the data given Excel's row limit (~1m). If you have problems using this file, please get in touch and I will send it via WeTransfer.

/// Prices. (5% sample). This is a 5% sample of the data, created to ease sharing and to get a feel for the data. If you get stuck with the large file above, please run your code on this. The variable names and definitions are identical, so you can work on it while I get you the big file.

/// Item. Information on the consumer products: their full names, descriptions, when they appear and exit the CPI, and the number of observations for each.

/// Region. The location the price quote was collected.

/// Date. The date on which the price quote was collected. Date is recorded in various formats to make it compatible with different types of software.

If you use the data and have comments on how to make it more useful, please get in touch.

Teaching examples

Some examples of courses that use the data as a teaching resrouce:

/// LSE. The Economics for Python weekend designed and taught by Rahat Siddique.

/// Imperial. The Data Stories course with Ralf Martin.

/// Bristol. The Communicating Economics course run with Sarah Smith and Christian Spielmann.

/// UCL. The Data Skills Lab with Parama Chaudhury.

Examples: interactive charts

Number of observations - by region

The ONS collects prices in proportion to the population in each of the NUTS1 areas. The pattern of useable prices (i.e post cleaning) that are available in the LRPD are below.

The Covid-19 pandemic disrupted price collection severely. Another version of the same chart, this time drawn for 2018-2021 allows us to focus on the recent volatility. Click on the legend to select a region and isolate its data.

Example - interactive chart: inflation vs volatility

The chart below plots cumulative inflation, by item, against the volatility of that item's prices. Some items change price a lot but the pattern of rises and falls generates little long-run inflation (e.g. lettuce, cucumber, sprouts, strawberries). Others change price infrequently, but these changes build over time to create large inflationary changes (e.g. haircuts, fish and chips, cheese).

The chart below is interactive. Try clicking on the legend to select the type of good you are interested in. Then hovering over dots to get more information. You can zoom ina and out on croweded areas of dots using your mouse or tracker ball. Coded in Vega Lite.

Example: animation: pints of bitter

The humble pint of bitter is one of the most sampled prices in the UK CPI. Here is how we went from 80p a pint in 1988 to a huge range of prices, inlcuding £6 a pint today. Note the peak that emerges at £2, which a knowledgeable drinker tells me is the 'Wetherspoons Effect'. Creating using simple stop frame animation (running a loop to create 395 monthly charts).