The probate fee dataset

What it is

Since 2018 every firm in England and Wales that does probate work for the public has been required to publish what it charges. Those prices are public, and they are also useless as a comparison, because they sit one firm at a time across thousands of separate websites in thousands of different formats.

This dataset is those pages read and made comparable. Each record is one firm, its SRA number, and every probate price it publishes, classified by what the price covers and how it is charged. Each price keeps the sentence the firm wrote around it, the address of the page it was read from, and the date it was read.

How it is built

The list of firms comes from the SRA register. The prices do not: they come from the firms' own websites, so the dataset does not depend on any one organisation continuing to supply it. Only firms the register shows as practising and doing probate work are read at all.

The crawler reads one page at a time per website with a gap between requests, honours robots.txt, identifies itself honestly and caps what it will download. Every price is re-read weekly, and a record we have not confirmed in 180 days is dropped rather than shown. A separate archive records every change, appended and never rewritten, because anybody can read today's prices and nobody can go back and read last year's.

The full method, including what the parser had to learn from real published pages and where it still fails, is written up in plain English.

What it does not cover

Firms that publish no readable price get no record. A firm whose price page we cannot read, or that publishes its prices only as a scanned image, is a gap in the coverage rather than a firm charging nothing, and the crawl report records the reason for every firm that did not make it.

It covers uncontested probate only. Contentious work sits outside the transparency rules, so firms are not required to publish a price for it and most do not. It also covers England and Wales only, because Scotland and Northern Ireland have different processes and a different regulator.

Licence

The dataset is published under CC BY 4.0. Reuse it, publish it, build on it and train on it. The condition is attribution, and attribution means naming Probate Panel and linking the page you took it from.

Covers the structured fee records and their compilation. The sentence quoted with each fee remains the firm's own copyright, reproduced with a link to the page it was published on.