An independent forum for academic debate on the challenges faced by contemporary society

Special issue on research methods and methodology: editorial

One of the original aims of People, Place and Policy Online was to encourage discussion about the relative strengths and weaknesses of different methodological positions and technical approaches to policy-orientated research and evaluation. This special issue presents four diverse contributions to this debate.  Important questions are raised about how research evidence may inform policy (Gilbertson and Wilson, and Jones), the utility of historical oral accounts (Batty) and the assumptions inherent within comparative international research (Gribat).

  • Full article

    There is little doubting the importance of exploring the relationship between research and policy making. New Labour came to power in 1997 championing the benefits of evidence based policy making (EBPM), a vision articulated by David Blunkett in 2002, who, echoing the words of Tony Blair in the 1997 Labour Party Manifesto, claimed that:

    This Government has given a clear commitment that we will be guided not by dogma but by an open-minded approach to understanding what works and why. This is central to our agenda for modernising government: using information and knowledge much more effectively and creatively at the heart of policymaking and policy delivery. David Blunkett, Speech to the ESRC (2 February 2002)

    EBPM provided New Labour with a framework that guided policy decisions, and granted significant rhetorical power. Of course, EBPM has been challenged (see for example Sanderson 2002, and Pawson 2002). It has also evolved: from the commissioning of large scale evaluations of major government programmes (e.g. Sure Start and New Deal for Communities), to a focus on evaluating pilot programmes and undertaking various forms of evidence reviews and the more recent rise of the expert review. In the UK, since 1997 there has also been a gradual shift in focus from learning how policies work to estimating their economic and social impact, partly reflecting the political cycle and the need of government to demonstrate outcomes in national political debate and to the electorate. The UK, along with other countries, has also witnessed the growth of increasingly sophisticated performance management and target regimes. These can be seen as a means through which central government directs policy delivery and as providing the basis for a new relationship between central and local government. In this context, the measurement of target achievement and the validity of measurement methodologies assumes real importance.

    People, Place and Policy Online welcomes empirically and theoretically informed discussion from different viewpoints examining the methodological and technical challenges, as well as the practical and political realities, of delivering policy orientated research and evaluation within this policy context.  Rising to this challenge, the articles in the special edition by Gilbertson and Wilson, and Jones, cast new empirical light on the validity of particular evaluation methodologies. Gilbertson and Wilson show that the measurement of volunteering (a key component of government agendas around citizenship) can vary greatly depending on how survey questions are asked and how much information is given to survey respondents to answer questions. Jones explores whether it is possible to derive measures of local job quality which enable a clearer assessment to be made of economic development, not least using measures such as the government’s preferred measure of Gross Value Added. Jones argues that measuring job quality may provide superior insights into understanding spatial economic inequalities.

    The other two articles in this edition focus more squarely on methods and methodology. Batty explores issues around history and memory in social research. This area of research has been subject to considerable criticism, not least by Hobsbawn (1997), who contrasts the value of diary accounts with the inexactitudes of memoire and its reliance on remembered events. Batty questions this presumed weakness, drawing on her experience of employing a visual method (a concept diagram) to explore relationships between remembered events. However, recognising the potential for such devices to prioritise the interests of the researcher over the researched, she highlights the need for a clear methodological framework to support the exploration of personal histories, whether in the near or distant past.

    Finally, Gribat reflects upon research in eastern Germany to consider the relationship between translation and power. She argues that, although research is increasingly assumed to be undertaken to international standards and norms, findings and theories presented in publications in English play a predominant role in structuring research agendas and defining what counts as ‘strong’ theory. Of course, non Anglophone concepts and terms do gain currency; for example Gemeinschaft and Geselschaft (from German) and terms around territorial cohesion and social cohesion (from French, but translated and mediated into European Union policies). However, Gribat argues that it is difficult to translate the specific cultural and social meaning of such terms and concepts. This point is made through reference to the example ofschrumfende Städte – shrinking cities – the translation of which into English, she argues, fails to convey the full meaning of loss brought about by large scale population migration in the context of German unification. However, Gribat concludes by arguing that such differences arising from translation should be considered an opportunity rather than a problem, providing a space for conceptual and political deliberation, within which resistance against the Anglophone hegemony might be possible.

    The four articles published in this special issue explore very different matters of method and methodology. However, they have in common a willingness to challenge accepted norms. In doing so, they pose questions of salience to all researchers involved in the practices of policy research. We very much hope that they prove to be the first of many articles in People, Place and Policy Online to critically examine research and evaluation methods and methodologies.

    References

    Blunkett, D. (2000) Influence or irrelevance: can social science improve government. Speech to the Economic and Social Research Council, 2 February.

    Hobsbawn, E. (1997) On History.  London: Weidenfeld and Nicholson.

    Pawson, R. (2002) Evidence-based Policy: In Search of a Method. Evaluation, 8, 2, 157-181. CrossRef link

    Sanderson, I. (2002) Making Sense of ‘What Works’: Evidence Based Policy Making as Instrumental Rationality? Public Policy and Administration, 17, 3, 61-75. CrossRef link