Skip to main content

https://dluhcdigital.blog.gov.uk/2024/01/29/changes-to-the-energy-performance-certificates-open-data-service/

Changes to the Energy Performance Certificates open data service

An example of an energy performance certificate

The Energy Performance Certificates (EPC) open data site was launched in 2016. It provides both an interface and programmatic access to allow users to search for and download almost 30 million energy performance certificates in England and Wales. The site is well-used, and experiences high volumes of activity, with 30 million API requests and 40,000 zip files downloaded each month. 

Over the years, we have been making incremental improvements to the service, and we are pleased to announce another round of improvements applied in January 2024.  

How we chose what to focus on

To determine which areas to focus on when making improvements, we have taken into account feedback from our users, analysis of usage statistics and a review of support tickets. We have also taken the opportunity to review the site through the lens of the latest design standards, as well as implementing features and approaches to getting data from other sites. 

From this analysis, we decided that there were 2 parts of the service that we wanted to improve – both centred around the way that users can get data out of the application.  

How users get data about EPCs 

The application already provides 3 ways for users to get data about energy performance certificates: searching in the application and viewing certificate data on screen, bulk downloading EPC data in CSV files, and accessing data programmatically via the API. Our analysis revealed that the bulk downloads and API access are both well-used, but support tickets and helpdesk enquiries indicated we could make them better. 

The improvements we’re making

Improvements to the bulk download part of the service include better visibility of the bulk downloads and more cuts of the data. Improvements to the API include: better documentation, with code snippets for each of the API calls we provide, a better way to get all the data our users need in one go, and application of the OpenAPI standard, giving users access to tools to support better exploration of the API through the new Swagger interface. 

Improving the bulk downloads 

While the service has supported bulk downloads of certain cuts of the data for some time, we have improved visibility of the bulk downloads and introduced new bulk download options. 

Previously, it was possible to download zip files containing all certificates and recommendations for the 3 certificate types (domestic, non-domestic and DEC), with files organised by local authority, as well as downloads for each local authority in England and Wales. 

We have now introduced the ability to download all certificates and recommendations in one file for each of the certificate types. We have also introduced downloads by specific time periods: monthly for each of the last 12 months, and then annually going back to 2008. 

Our research indicated the existing local authority downloads were not sufficiently obvious to users, so we have created 3 new pages to provide full lists of all the available files for downloading (all certificates and recommendations, all certificates and recommendations by local authority, and all certificates and recommendations by time period). Find out more about domestic EPCs.

Some of the downloads now have a schema.json file available within the zip bundle to provide machine-readable information about the contents of the files. 

All of the download files have a predictable, permanent URL so if you wish, you can programmatically access the files. 

Improving the API and documentation 

Previously, the API provided access to data within the site, but individual requests were limited to a page size of 5,000 records, with an overall limit of 10,000 records. Feedback from users suggested that this limitation makes it difficult to work with the data, requiring multiple requests, applying different filters to build the dataset. 

We have reconfigured the underlying database to remove this limitation, so API requests can now provide access to all of the data provided through the service, using a new pagination method. We will be closely monitoring the platform to ensure that there is no degradation in performance for our users. To help reduce the load on the server, please make use of the filters to download the data you need. 

In addition to this, we now provide OpenAPI v3 schemas for all of the available APIs. This is a way of describing the APIs in a standard way. Using the OpenAPI standard means we have been able to create a Swagger interface to support developers with building their API calls. We have also updated the documentation to include code snippets in cURL and Python for each API call to make it easier for users to access data programmatically. You can see the updated documentation here

Feedback 

We will be closely monitoring the platform to ensure that there is no degradation in performance for our users. To help reduce the load on the server, please make use of the filters to download the data you need.

As always, we’d love to hear your feedback on these changes and any other features of the EPC service. Contact us by email at EPC.Enquiry@communities.gov.uk

Sharing and comments

Share this page

7 comments

  1. Comment by John Murray posted on

    The blog says we can programmatically download the zip files, but I tried by passing my API token in the GET headers, via Python as I do for the API, but it doesn't appear to work. I can't find how to pass authorisation headers for bulk downloads in the documentation. Please advise.

    Reply
    • Replies to John Murray>

      Comment by Steve Peters posted on

      Hello John

      Thanks very much for taking the time to feedback. You've (helpfully for us) found a bug in the APIs. Please accept my apologies for any issues this causes you.

      We're working on a fix now for release within the next 2 weeks. Thanks for bearing with us.

      Reply
  2. Comment by Andy Mitchell posted on

    CONSTRUCTION AGE BAND

    CONSTRUCTION_AGE_BAND VARCHAR 31
    Age band when building part constructed. England & Wales only. One of: before 1900; 1900-1929; 1930-1949; 1950-1966; 1967-1975; 1976-1982; 1983-1990; 1991-1995; 1996-2002; 2003-2006; 2007-2011; 2012 onwards.

    The API is returning different values to that described here, for example:
    "England and Wales: 1967-1975"

    Please update your documentation.

    Reply
    • Replies to Andy Mitchell>

      Comment by Steve Peters posted on

      Thanks Andy for taking time to feedback. We will update the documentation ASAP.

      Reply
  3. Comment by Andrew lam posted on

    Dear Admin, I am trying to access the website for this link, but I find that I am not able to create an account to sign up even after entering my data. Would it be possible to assist with this enquiry

    Reply
    • Replies to Andrew lam>

      Comment by Steve Peters posted on

      Thanks very much for your enquiry. We have checked the service, and believe it is issuing login credentials correctly. You should have received an e-mail with a link to activate and use your login. Could you check whether the e-mail has been directed to your junk messages folder please?

      If you have the message but the login still isn’t working, please do come back to us with more detail – including screenshots – of the problem.

      Reply
      • Replies to Steve Peters>

        Comment by Andrew Lam posted on

        Dear Steve, thanks, it actually worked around 1 hour after I signed in. Thank you for the help. God bless.

        Reply

Leave a comment

We only ask for your email address so we know you're a real person

By submitting a comment you understand it may be published on this public website. Please read our privacy notice to see how the GOV.UK blogging platform handles your information.