When I joined the Guardian in 1969 there were only three social policy reporters – one covering health and social care, one on education and a third on housing. By the mid-noughties there were 30 – which included journalists on the Society and Education supplements and the Guardian’s website.

What led to such a vast expansion of social policy reporting? First was the expansion of public services. By 2004/5 spending on social policy – health, education, housing, social security and social services – had increased sevenfold in real terms since 1950. then there was the rise in political importance of these services. The message from opinion polls in the late 1990s was clear: invest more in public services rather than tax cuts. a third factor was that social policy stories had multiple human interest angles. And fourth, in terms of the Guardian, was that social policy reporting was generating large amounts of resources from job advertisements in its specialist sections.

The growth of specialist reporters across Fleet Street changed the nature of ministerial press briefings. In the 1970s ministers could frequently get away with not knowing very much, as many meetings were attended by general reporters. With the development of specialist reporters, the balance of knowledge changed. Specialist journalists often knew more than ministers. Hence the emergence of pre-scripted statements and more tightly controlled question sessions.

One factor holding back social policy reporting when I joined the paper was a lack of access to crucial information. there were few inspectors of public services and none produced detailed reports; little public audit; no Freedom of Information Act; no specialist committees in parliament. But over the next four decades the position was transformed. Inspectorates mushroomed, producing detailed assessments of institutions and reports addressing current problems and future challenges. The Audit Commission, set up in 1983, opened up councils and led to improvements. a beefed up National Audit Office, parliament’s watchdog, did the same for Whitehall’s departments, along with the emergence of select committees and their special advisers. And in 2005, a Freedom of Information Act was finally in operation.

British government, central and local, was opened up. it could have been a golden period for journalism but Fleet Street was guilty of serious shortcomings which I highlight in Democracy Under Attack – How the Media Distort Policy and Politics, published yesterday . they include dumbing down, distortions, hunting in packs, and concentrating on the negative.

The idea that newspapers can hold governments to account was, I believe, always a professional myth. The numbers are too big: 25,000 schools, thousands of care homes, hundreds of hospitals. Only inspectors can do that. But in our random way, we have helped correct wrongs. I fear there will be much less of this in the future. When I began the research for my book the mainstream media was still widely regarded as a big, bad uncontrolled beast. it ends with newspapers widely seen as fatally wounded, leaching at an accelerating rate readers, viewers and advertisers to the web. Papers have shrunk, specialist reporters have been slashed and profits have disappeared.

As a result there will be fewer well-informed journalists to analyse and present the increasing amount of data. fewer specialists also means fewer awkward questions asked in ministerial briefings. Less grit in the democratic oyster means fewer policy pearls.

Malcolm Dean was the founding editor of Society.

<a href="http://www.guardian.co.uk/society/joepublic/2011/nov/08/social-policy-specialist-journalists?newsfeed=truetag:news.google.com,2005:cluster=http://www.guardian.co.uk/society/joepublic/2011/nov/08/social-policy-specialist-journalists?newsfeed=trueWed, 09 Nov 2011 00:16:32 GMT”>How journalists have shaped social policy