This page houses my work on prices and inflation, using 'micro data' to investigate inflationary puzzles and implications. I update the page monthly, my latest discussion note is here, and working paper is here.
The UK has great price data, with official CPI 'micro data' released with a lag of only one month, allowing work on prices, inflation and real wages to be conducted in (close to) real time. I have used this to create a long-run prices database (LRPD), which make available here.
To get started, select an item using the drop-down box below. The prices that underly our official inflation measures behave in interesting, and very different, ways.
Papers and ongoing research
Some papers and blogs using this data include:
/// Prices in the UK, a new dataset. Occasional Paper setting out the LRPD. CEP. (Data Annex here)
/// The changing cost of Christmas. An interactive piece, demonstrating the data. Economics Observatory.
This is ongoing research, if you have any comments or would like to use the data, please get in touch.
Research and teaching - examples
Some examples of courses and websites that have used the data either as a teaching resource or as an analytical tool:
/// LSE. The Economics for Python weekend designed and taught by Rahat Siddique. In the latest iteration of this, "Team BoE" put together some intersting new indices. You can get their presentation here, and Google Colab code (Pyhton code, running in the browser) here.
/// Imperial. The Data Stories course with Ralf Martin.
The long run prices database - using the data
I update the LRPD monthly, the latest update was 20th Oct 2021. The data is stored in the following files (all csv):
/// Prices. (Full database). This contains the price data. It is a very large file (there are currently ~41m observations.) To conserve memory all supplementary details are kept in other databases that can be linked to when needed. Update: lots of people are opening this in Excel and thereby truncating the data to 1m observations - please get in touch if you want the file and I will send it directly.
/// Prices. (5% sample). This is a 5% sample of the data, created to ease sharing and to get a feel for the data. If you get stuck with the large file above, please run your code on this. The variable names and definitions are identical, so you can work on it while I get you the big file. Too large to open in Excel.
/// Deciles. This file allows you to track the distribution of prices. It is the 10th to 90th percentiles inclusive in decile steps, for each item in each month. Around 200k observations - can be used in Excel.
/// Region. The location the price quote was collected. Small file.
/// Date. The date on which the price quote was collected. Date is recorded in many formats (dd/mm/yyyy, yyyy-mm-dd, etc) to make it compatible with different types of software. Small file.
If you use the data and have comments on how to make it more useful, please get in touch.
Examples: interactive charts
Number of observations - by region
The ONS collects prices in proportion to the population in each of the NUTS1 areas. The pattern of useable prices (i.e post cleaning) that are available in the LRPD are below.
The Covid-19 pandemic disrupted price collection severely. Another version of the same chart, this time drawn for 2018-2021 allows us to focus on the recent volatility. Click on the legend to select a region and isolate its data.
Example - interactive chart: inflation vs volatility
The chart below plots cumulative inflation, by item, against the volatility of that item's prices. Some items change price a lot but the pattern of rises and falls generates little long-run inflation (e.g. lettuce, cucumber, sprouts, strawberries). Others change price infrequently, but these changes build over time to create large inflationary changes (e.g. haircuts, fish and chips, cheese).
The chart below is interactive. Try clicking on the legend to select the type of good you are interested in. Then hovering over dots to get more information. You can zoom ina and out on croweded areas of dots using your mouse or tracker ball. Coded in Vega Lite.
[Chart currently being updated]
Example: animation: pints of bitter
The humble pint of bitter is one of the most sampled prices in the UK CPI. Here is how we went from 80p a pint in 1988 to a huge range of prices, inlcuding £6 a pint today. Note the peak that emerges at £2, which a knowledgeable drinker tells me is the 'Wetherspoons Effect'.
The video is created using simple stop frame animation (running a loop to create 395 monthly charts).