This page houses my work on prices and inflation, using 'micro data' to investigate inflationary puzzles and implications. I update the page monthly. Discussion notes and working papers are here and here (fuller list below). I have put together an open-source GitHub repository with prices and inflation analysis. This is at https://rdeconomist.github.io/inflation.
The UK has great price data, with official CPI 'micro data' released with a lag of only one month, allowing work on prices, inflation and real wages to be conducted in (close to) real time. I have used this to create a long-run prices database (LRPD), which I make available here and update each month. The latest update was April 2024
To get started, select an item using the drop-down box below. The prices that underlie our official inflation measures behave in interesting, and very different, ways.
Papers and ongoing research
Some papers and blogs using this data include:
/// Bakker, Datta, Davies, de Lyon (2022). Non-tariff barriers and consumer prices: evidence from Brexit, CEP Discussion Paper.
/// Watson, Jacob and Smith (2022). The rising cost of living and its effect on Londoners. Greater London Authority. Covered in the Evening Standard, and City AM.
/// de Bakker et al (2022). Imports, supply chains and consumer prices. Discussion paper, UK in a Changing Europe.
/// Davies (2021). Prices in a pandemic. Discussion note. CEP. Covered in The Times (London), the Financial Times and the FT's Alphaville blog.
/// Davies (2021). Prices in the UK, a new dataset. Occasional Paper setting out the LRPD. CEP. (Data Annex here)
/// Davies (2020). The changing cost of Christmas. An interactive piece, demonstrating the data. Economics Observatory.
This is ongoing research, if you have any comments or would like to use the data, please get in touch.
Research and teaching - examples
Some examples of courses and websites that have used the data either as a teaching resource or as an analytical tool:
/// LSE. The Economics for Python weekend designed and taught by Rahat Siddique. In the latest iteration of this, "Team BoE" put together some interesting new indices. You can get their presentation here, and Google Colab code (Python code, running in the browser) here.
/// Imperial. The Data Stories course with Ralf Martin.
/// Bristol. The Communicating Economics course run with Sarah Smith and Christian Spielmann. DropBox.
/// UCL. The Data Skills Lab with Parama Chaudhury.
/// Economics Observatory. Analysis of the UK's 'Tampon Tax' by Sarah Smith, and related piece by Gemma Williams.
The long run prices database - using the data
I update the LRPD monthly. The data is stored in the following files (all csv):
/// Prices. (Full database). This contains the price data. It is a very large file (there are currently ~43m observations.) To conserve memory all supplementary details are kept in other databases that can be linked to when needed. Update: lots of people are opening this in Excel and thereby truncating the data to 1m observations - please get in touch if you want the file and I will send it directly.
/// Prices. (5% sample). This is a 5% sample of the data, created to ease sharing and to get a feel for the data. If you get stuck with the large file above, please run your code on this. The variable names and definitions are identical, so you can work on it while I get you the big file. Too large to open in Excel.
/// Deciles. This file allows you to track the distribution of prices. It is the 10th to 90th percentiles inclusive in decile steps, for each item in each month. Around 200k observations - can be used in Excel.
/// Item. Information on the consumer products: their full names, descriptions, when they appear and exit the CPI, and the number of observations for each. Small file.
/// Region. The location the price quote was collected. Small file.
/// Date. The date on which the price quote was collected. Date is recorded in many formats (dd/mm/yyyy, yyyy-mm-dd, etc) to make it compatible with different types of software. Small file.
If you use the data and have comments on how to make it more useful, or if you need Stata files, item weights etc, please get in touch.
Examples: interactive charts
Number of observations - by region
The ONS collects prices in proportion to the population in each of the NUTS1 areas. The pattern of useable prices (i.e post cleaning) that are available in the LRPD are below.
The Covid-19 pandemic disrupted price collection severely. Another version of the same chart, this time drawn for 2018-2021 allows us to focus on the recent volatility. Click on the legend to select a region and isolate its data.
Example - interactive chart: inflation vs volatility
The chart below plots the CPI microdata to allow users to zoom in and investigate items of interest. The colours of the dots represent the COICOP division (roughly, the type of good), the size of the dots is the median price in the sample (i.e. between 1988 and the latest data).
The chart is interactive. Try clicking on the legend to select the type of good you are interested in. Hover over dots to get more information on each item. Zoom in and out on crowded areas of dots using your mouse tracker ball. The chart is coded in Vega Lite and the chart spec is available on GitHub.
Example: animation: pints of bitter
The humble pint of bitter is one of the most sampled prices in the UK CPI. Here is how we went from 80p a pint in 1988 to a huge range of prices, including £6 a pint today. Note the peak that emerges at £2, which a knowledgeable drinker tells me is the 'Wetherspoons Effect'.
The video is created using simple stop-frame animation (running a loop to create 395 monthly charts).