How the fee dataset is built
A polite crawler reads each firm's published price page, a parser reads the figures, and every record keeps its evidence and its date.
The list of firms comes from the SRA register. The prices do not: they come from each firm's own website, published because the SRA Transparency Rules require it. So the dataset does not depend on any single organisation continuing to supply it.
The crawler reads one page at a time per website, with a gap between requests, honours robots.txt, identifies itself honestly and caps how much it will download. It is reading the websites of the firms we list, and it behaves accordingly.
The parser is the part that decides whether this site can be trusted. Firms publish a heading, then a label, then a bare amount, so context has to be carried down the page. One page usually covers every service a firm offers, so a conveyancing price under a probate heading has to be recognised as a conveyancing price. A rate card reads as a list of very cheap fixed fees unless it is understood as rates, and a cheap wrong number sorts straight to the top of a price comparison.
Every price is re-read weekly. A record we have not been able to confirm in 180 days is dropped rather than shown, and the build fails rather than publishing a figure nobody has checked in half a year. Every change is recorded in an archive that is only ever appended to, because anybody can read today's prices and nobody can go back and read last year's.