An independent forum for academic debate on the challenges faced by contemporary society

Measuring participation at a local level: be careful what you ask for!

Summary

In light of increasing interest in and need for evidence of participation at the local level, this article discusses methodological issues associated with measuring social participation and volunteering. Measuring these subjects is not straightforward and the methodology employed in surveys plays a part in influencing the levels of involvement reported. At a national level, surveys of volunteering and the Government’s Citizenship Surveys offer an established way of asking about volunteering. There is much less agreement on how to ask about volunteering at a local level. There is a need to validate findings from local surveys to explore which questions might be the most appropriate for local surveys aiming to measure participation levels.

  • Full article

    Introduction

    This paper reports on an Economic and Social Research Council (ESRC) funded project which aims to explore methodological issues associated with measuring participation at a local level. The research is timely as local authorities are now required to report on volunteering targets set out in Local Area Agreements (LAAs) and reliable local data is increasingly needed. The research used qualitative interviews to repeat survey questions on participation in organisations and groups with respondents from a previous local survey to test out the questions and to validate survey responses. These interviews provide insight into respondents’ understanding of the questions, and help to highlight possible shortcomings in survey questions aimed at measuring involvement in groups and organisations. Finally, in light of issues raised the paper discusses some concerns for researchers and policy makers.

    Context

    Over the past decade social participation and volunteering have received considerable political attention. The Labour Government has invested money in a variety of initiatives to promote volunteering (see Low et al., 2007) and regard increasing levels of participation and volunteering as indicators of strong communities. In particular, the Government has promoted community involvement as part of its civil renewal agenda (see Lowndes et al., 2001; Brannan et al., 2006), and encouraged active participation in communities through numerous initiatives across a range of policy areas (Williams, 2003).

    Participation and volunteering are also now key targets set by the Government and are included in a set of National Indicators (NIs) which reflect national priority outcomes. The Local Government White Paper Strong and Prosperous Communities (CLG, 2006) introduced a set of 198 National Indicators which are included in the local government performance framework and cover all the national priority outcomes which local authorities are responsible for delivering. Two national indicators in particular are concerned with participation and volunteering – NI 3 ‘civic participation in the local area’ and NI 6 ‘participation in regular volunteering’ (CLG, 2008).

    Levels of participation and volunteering are monitored and evaluated primarily through the series of national Citizenship Surveys (see Attwood, 2003; Green et al., 2004; Kitchen et al., 2006). Other large national surveys like the General Household Survey (GHS) and a number of national surveys of volunteering also measure the prevalence of volunteering (see Field and Hedges, 1984; Lynn and Davis Smith, 1991; Davis Smith, 1998; Low et al., 2007). The surveys collect information on a variety of topics including participation in groups, clubs and organisations and volunteering.

    The UK does not have a common national definition of volunteering, and various definitions are used in reports, surveys and legislation. Usually definitions state volunteering involves unpaid activity where someone gives their time freely to help an organisation or an individual who they are not related to. Volunteering covers a wide range of activities and can be defined as:

    An activity that involves spending time unpaid, doing something that aims to benefit the environment or individuals or groups other than (or in some cases in addition to) close relatives(Home Office, 2005).

    Within this general definition there are different types of participation, and volunteering is most often split into informal and formal activity. Formal volunteering is classed as that which takes place within groups, clubs or organisations whereas informal volunteering is undertaken on an individual one-to-one basis to others who are not members of the family. According to Attwood et al., (2003) formal volunteering has its roots in social participation but involves greater commitment. Social participation is being involved in groups, clubs or voluntary organisations. Definitions of volunteering also distinguish the frequency of activity. Regular volunteering is usually defined as that which takes place at least once a month or more; occasional volunteering takes place less frequently than once a month; and one-off acts of volunteering are classed as episodic volunteering.

    On the whole, data on participation and volunteering are collected at the national and regional level, and there is much less information available at a local level. Some local information is obtainable following the release of data from the Place Survey in June 2009. Administered by local authorities, the Place Survey will be used to measure a number of NIs set out in the local government performance framework including NI 3 and NI 6. Data will be used by local authorities and other partners to monitor progress on volunteering targets set out in some LAAs. LAAs are at the heart of the performance framework for local authorities and are the means for agreeing shared targets with central government. Each LAA will have up to 35 national priority targets with specific improvement targets set for each.

    Although there are different surveys gathering information on levels of participation and volunteering, these topics remain difficult to measure and comparison of figures between different surveys is not straightforward. Research methodology employed in surveys can influence the levels of volunteering and participation reported in studies.  The definition of what is being measured, the technical aspects and the scope of studies all play a part in determining the levels participation and volunteering reported (see Lyonset al., 1998). Even the phrasing and extent of questioning in surveys of voluntary activity have been shown to account for differences in reported levels of participation (see Evans and Saxton, 2005; Lynn, 1997). Lynn’s work which examined 11 surveys of voluntary activity in Britain between 1976 and 1990 concluded that the main reason for the difference in the numbers reporting participation between the studies was the extent of the questioning. The more time and more prompts people were given resulted in higher estimates of participation in voluntary activity. Lynn also believed that responses were higher in surveys that did not use the terms ‘voluntary work’ or ‘voluntary activity’. A body of research from the USA/Canada examining the effect of methodology (see Steinberg et al., 2002; Rooney et al., 2004) and other methodological considerations (see Hall, 2001; O‘Neill, 2001) on reported levels of volunteering also supports such findings.

    Despite the methodological problems associated with measuring participation and volunteering, through the series of national surveys of volunteering and the Citizenship Surveys there is a fairly well established way of asking about volunteering at a national level. In the case of formal volunteering, these surveys tend to use a system of extended questioning with prompts to establish participation in a range of different groups, clubs and organisations. Once involvement in any of the groups, clubs or organisations is determined, and in order to assess whether it can be defined as formal volunteering, the surveys ask whether involvement in the particular groups, clubs and organisations picked out has involved any of a wide variety of listed activities which have been defined as volunteering. Surveys which can devote only a limited amount of questionnaire space and interviewer time to the topic use shorter questions. They tend not to ask about participation in a list of different types of groups, clubs and organisations and do not provide examples of activities which are classed as volunteering. The Place Survey for example, employs only one question to establish regular volunteering, does not provide categories of groups, clubs and organisations for respondents to choose from and conflates both formal and informal volunteering into the question.

    There is much less agreement on how to ask about volunteering at a local level. One reason is it is not really feasible to repeat national surveys at a local level. Such an exercise would be prohibitively expensive. Instead local surveys have developed in an ad hoc way and have failed to develop consistency when asking about participation and volunteering often resulting in data which is incomparable with either national figures or to other local areas.

    Those wanting to collect information at a local level are faced with a number of problems. First of all there is a dearth of reliable information on participation levels at a community level. Secondly, small scale surveys are likely to have limited time and resources to ask a long series of questions with many prompts. Incorporating an extended set of questions on participation and volunteering typically used in established national surveys will be difficult. Thirdly, comparing reported levels of participation and volunteering with other studies will be problematic and at the very least will require an awareness of how the research design, methodology and questioning methods employed in a study may influence results. Finally, limited evidence is available on how to measure participation at a local level in a way which is both methodologically sound and practically feasible. Given these concerns, information and evidence on how well questions have worked in previous local studies is likely to be useful. Research is needed to validate responses from local surveys and to explore which questions might be the most appropriate for local surveys aiming to measure participation levels.

    The South Yorkshire Social Capital Survey

    The data used in this research was originally collected as part of an earlier longitudinal study which explored social capital in nine ex-coalfield communities in South Yorkshire in 2000 and 2004 and examined the relationship between social capital and health, and social capital and social participation (Green et al., 2000; Gilbertson et al., 2005; Gilbertson and Manning, 2006a; 2006b).

    The South Yorkshire Surveys in 2000 and 2004 measured levels of participation in groups and local organisations. The survey used two questions from the social capital module which was being piloted by the Office of National Statistics for use in the next GHS at the time. These questions were chosen for the South Yorkshire study to take advantage of the developmental work around defining and measuring the concept of social capital. Based on the following definition of social capital: ‘networks together with shared norms, values and understandings that facilitate co-operation’ within or among groups’. (Cote and Healy, 2001: 41), the social capital questions built on the typology developed by Blaxter et al.(2001), the GHS and the Citizenship Surveys.  Five dimensions of social capital were identified, one of which was social participation or involvement in groups, clubs and organisations. The full set of harmonised social capital questions (ONS, 2008, version 3), are now incorporated in a number of government surveys including the GHS. Cross governmental harmonisation has standardised a number of definitions, concepts and questions used in the wide range of government surveys to make it easier for users to draw robust comparisons between different data sources.

    The first question used in the South Yorkshire Survey was: Have you been involved in any local organisation(s) over the last three years? In 2004, a second question was added to the survey which asked respondents to select from a list of groups any they had taken part in over the last three years. This question aimed to gain a more detailed understanding of participation in these areas. The questions employed in the South Yorkshire Surveys were similar to questions in the GHS in 2000 and 2004/5 (see Figure 1), and despite differences provide the best benchmark data available.

    The differences between the GHS questions and those used in South Yorkshire were mainly due to the need for the South Yorkshire Survey to be kept relatively short so as not to alienate an already over-surveyed population. The project also trained and employed residents to undertake interviews and the survey unit responsible advised that inexperienced resident interviewers were likely to be more successful with a simple user friendly questionnaire. By 2004/5 the timeframe of the GHS questions had changed from three years to twelve months. However, in order to maintain consistency and ensure both waves of the South Yorkshire Survey were comparable, it was decided that the timeframe for the questions in the 2004 survey should be kept to three years.

    The questions included in the GHS ask about involvement in three types of groups, informal, local and national. The three questions act as probes and although they are repetitive to respondents, this approach increases the number of groups reported (see Green and Fletcher, 2003). The GHS is similar to the Citizenship Surveys and national surveys of volunteering in that they ask respondents to pick out groups, clubs or organisations that they have taken part in over the last 12 months. However crucially, in the 2004/5 GHS, this is preceded by a yes/no filter question: respondents only get asked to pick out types of groups if they have stated ‘yes’ to having taken part in a group. The list of the types of groups is also shorter in the GHS and does not include categories for safety and first aid, justice and human rights, citizens group or two separate categories for children’s education and young people’s and children’s activities outside schools. These differences are likely to result in lower estimates of social participation in the GHS to those reported in either of the other surveys.

    Figure 1: Survey Questions

    South Yorkshire Social Capital Survey, 2000

    Have you been involved in any local organisation(s) or activities over the last 3 years? (Such as sports, PTA, youth clubs)  (Yes / No)

    South Yorkshire Social Capital Survey, 2004

    Have you been involved in any local organisation(s) or activities over the last 3 years? (Such as sports, PTA, youth clubs)  (Yes / No)

    Which of these categories on this card best describe any groups you have taken part in over the last 3 years? 

    • Hobbies/social clubs
    • Sports/exercise groups, including taking part, coaching or going to watch
    • Local community or neighbourhood groups
    • Groups for children or young people
    • Adult education groups
    • Groups for older people
    • Environmental groups
    • Health, disability and welfare groups
    • Political groups
    • Trade Union groups
    • Religious groups, including going to a place of worship or belonging to a religious based group
    • Other group (please specify).

    General Household Survey, 2000

    Have you been involved in any local organisation over the last three years? (Yes / No)

    General Household Survey, 2004/05

    The next questions are about involvement in groups, clubs and organisations.  These could be formally organised groups or just groups of people who get together to do an activity or talk about things.  Please exclude just paying a subscription, giving money and anything that was a requirement of your job.

    I am going to ask about three different types of groups:

    First, in the last 12 months have you been involved with any groups of people who get together to do an activity or talk about things?  These could include evening classes, support groups, slimming clubs, keep-fit classes, pub teams and so on.  (Yes/No)

    Which categories on this card best describe the groups you have taken part in?  (Categories of responses provided are identical to those above).

    Second, in the last 12 months, have you taken part in any (other) group activities as part of a local or community group, club or organisation?  These could include residents’ associations, sports groups, parent-teacher associations, school or religious groups and so on.  (Yes/No)

    Which categories on this card best describe the groups you have taken part in?  (Categories of responses provided are identical to those above).

    And third, in the last 12 months, have you taken part in any (other) group activities as part of a national group, club or organisation?  These could include pressure groups, charities, political groups, environmental groups and so on (Yes / No)

    Which categories on this card best describe the groups you have taken part in? (Categories of responses provided are identical to those above)

    What Activity are the South Yorkshire and General Household Survey Questions Measuring?

    Instructions held in the data archive (ONS, 2006) indicate the 2004/5 GHS questions on social participation should be summed to give an overall measure of involvement.  As such, it is feasible they capture some participation which could be classified as formal volunteering too.  Questions in the 2004/5 GHS and the South Yorkshire Survey do not ask about the level of responsibility or commitment involved and, therefore, it is not possible to ascertain what activity reported by respondents could potentially be classified as formal volunteering.  To differentiate volunteering from social participation, the 2004/5 GHS follows the questions outlined in Figure 1 with questions specifically about unpaid help to any groups, clubs and organisations (formal volunteering) as well as questions on help to individuals who do not live with respondents (informal volunteering).  In the GHS the extent of both formal and informal volunteering can then be reported separately to that of participation in groups, clubs and organisations.  Used alone the questions in the South Yorkshire Survey do gather information on a wide range of social participation activities but do not provide any data on the type or quality of the involvement in local organisations or groups.

    Levels of Involvement in Groups and Organisations Reported in the South Yorkshire Survey and the General Household Survey

    Although the differences in questioning outlined in Figure 1 complicate comparison, there will be some overlap between what the questions employed in the South Yorkshire Survey and those in the GHS 2004/5 are measuring. Levels of involvement in local organisations reported in both surveys are similar and this may indicate that these questions are capturing similar information. In 2004, 23 per cent1 of South Yorkshire respondents reported being involved in a local organisation compared to 24 of 2004/5 GHS who reported involvement in a local or community group, club or organisation. However, when responses to the three questions in the 2004/5 GHS are summed to give an overall measure of involvement and participation levels are compared to those participating in groups in South Yorkshire,  there are significant differences between the two (Figure 2).

    Figure 2: Levels of Social participation

    Figure 2: Levels of Social participation

    Note: * Differences are significant at the 0.05 level; ** differences are significant at the 0.01 level

    These variations may well be accounted for by differences in the survey populations, but it is possible the wording and approach to questioning may play a part in explaining some of the difference. The effect of the repetitive questioning in the GHS could contribute to the large difference in the proportion of respondents who report taking part in one or two groups. Uncertainty about whether the second South Yorkshire question captures information measured by all three of the GHS questions also makes a comparison of these figures more difficult to interpret.

    Evidence from Qualitative Interviews with Respondents who took part in the South Yorkshire Surveys in 2000 and 2004

    Re-examining the South Yorkshire Survey data for the ESRC study has involved 42 semi-structured qualitative interviews with respondents who took part in both surveys in 2000 and 2004. Unless stated otherwise, the results presented below are for 38 interviews which have been analysed in full. The interviews provided an opportunity to repeat the survey questions with respondents in order to ‘test out’ how they worked. Another purpose of the interviews was to discuss with respondents their understanding of the questions and ask them about their responses to the survey questions in 2000 and 2004.

    What activities did the questions capture?

    Responses to both of the questions employed in the South Yorkshire Survey included a wide range of activities. At the qualitative interview when respondents were asked the first question about involvement in local organisations and activities, answers included helping out at a local cubs group, membership on the committees of a local bowling club, play group and a model engineering society, membership of the local Labour party, being involved in a local National Trust group, being a trustee of a local leisure centre, attending a course at the local Sure Start and setting up a petition (under the activities part of the question). A list of responses is presented in Figure 3 and shows that 18 of 38 respondents said they were involved in a local organisation or activities.

    Figure 3: Responses to the South Yorkshire Local Organisation Question

    Have you been involved in any local organisation(s) or activities over the last 3 years?’

    • Member of bowling club and involved in the committee
    • Helps out at a local youth club
    • Involved in the committee of a local play group – bakes cakes to help raise funds 
    • Involved with a group of work colleagues and friends in raising money for local hospice
    • Attends keep fit at the local community centre
    • Trustee of local leisure centre
    • Member of the committee of a model engineering society
    • Organises matches and tournaments for the local junior football club
    • Helps out at the local cubs group
    • Attends Weight Watchers
    • Involved in setting up a local petition
    • Attends and helps out at local WI, Mother’s Union meetings, Amenities Society
    • Member of local Labour party
    • Governor at local primary school; also helps out at school as much as possible (reading once a fortnight); awareness raising and fundraising for SARAG (Sheffield and Rotherham Asbestos Group).
    • Member of local BNP (contributes financially)
    • Involved in local National Trust group and involved in a variety of local house groups through the church, also other activities organised through the church such as concerts
    • Attends Sure Start play group and other activities
    • Involved in the Stepping Out project (involved in activities such as fishing bowling etc.) organised by Headway (brain injury charity) local group

    The activities described above can be classified under three of the four types of community participation defined by Attwood et al., 2003 and cover social participation, civic participation and formal volunteering. At least 11 of the responses can be described as formal volunteering and are marked in bold. These activities (apart from setting up a petition) were then included under an appropriate type of group category listed in the second South Yorkshire question.

    Although the first question encapsulated a variety of activities it did not pick up any involvement in local social clubs or working men’s clubs, organisations which remained common in the communities surveyed. In six cases respondents who attended social clubs, some of them regularly, did not include the activity under the first question. An explanation can be found in how respondents regarded the question in the next section below. Furthermore, the exclusion of social club activity also seems to reflect how the clubs were largely utilised. Respondents often used the clubs for social drinking and whilst some had been active in the past, or were involved in societies and events organised through the social clubs, a number simply paid their annual subscriptions and visited occasionally. One respondent when asked why he had not included his membership of the local social club in the question commented that it was more of a ‘pleasure thing’ and not something he thought the interviewer was ‘asking about’ or would be ‘interested in’. Another respondent who had been involved in a sports and social club for 12 years and whose participation had included attending events and providing general help for several activities organised by the club, did not regard the social club as local to where she lived, and so, did not include her involvement in the first question. The relevance of the use of ‘local’ in the first question is discussed in the next section of the paper.

    How did respondents perceive the question on involvement in local organisations?

    A number of respondents viewed the first question as being about greater commitment, more direct and often regular involvement, being active, being one of the organisers, and even perhaps volunteering.

    Yeah being involved and ‘erm going every week to whatever meetings they’re having… if you get involved they expect you to go and you’re committing yourself aren’t you, you’re committing.  It’s a level of commitment that you’ve got to be there haven’t you?  You’ve said you’ll go, you’ll go.

    It’s probably that what I thought you were asking really was were I involved in organising events, being part of committees etc.

    I thought you meant what I helped out at, not what I’ve done generally.

    Such a distinction suggests this question implies something different to respondents than the question about groups and it may capture slightly different aspects of participation.  It is also possible the wording of the question influenced responses.  The first question asks about ‘involvement’ rather than ‘taking part’ and appears to suggest a greater degree of commitment to respondents.  There was also a sense that involvement in a local organisation might be about something more ‘formal’ as opposed to just going along to something as part of a group.  To a certain extent this understanding of the question is reflected in the activities respondents included in their answers, with over half of those who said they were involved in a local organisation listing a formal volunteering activity.

    Another issue raised by respondents was the ‘local’ aspect to the first question.  A number made a definite assessment of what they regarded as their ‘local’ area and excluded activities they then included in the second question on types of groups.

    I still say no again, I’m a member of the golf club, I’ve been a member of that club since 1962 and at that time when I joined it wasn’t part of this area, it were Wath on Dearne, now it’s part of Rotherham Borough, but I just don’t consider, same as going to see Rotherham United play football, it’s not local really.

    I wouldn’t say it’s in the local community but nearby.

    Is that why you didn’t mention it in the first question as well?

    Probably yeah because I don’t think of that as my local community.

    When asked about what respondents saw as ‘local’, for many it did not extend beyond a few streets from where they lived or outside of their communities.

    they’re not actually part of this community ‘cos the school she goes in isn’t in this area, it’s more or less in the town area’ … I’d place that as more in town’s community rather than our community (respondent has helped on a stall at daughter’s school summer fair and been to a coffee morning at the school which is less than a mile down the road).

    It is also worth noting the order of the questions in the South Yorkshire Survey and the use of ‘local’ in the first question but not in the second probably influenced responses.  By ordering the questions in such a way there was a risk the first question could potentially contaminate responses to the second.  Respondents may have thought the second question was asking solely about the local area rather than asking about involvement in any groups they took part in.

    Did the South Yorkshire Survey responses reflect levels of involvement?

    When asked about their survey responses most respondents felt their answers were a fair reflection of their participation at the time. Given respondents were being asked about previous responses they had made to a survey, it is possible they perceived the interviewer to be checking up on them and felt under some pressure to confirm their answers were correct. Also, the length of time since the surveys could possibly have made it more difficult for respondents to recall what they were doing at the time. Despite these limitations respondents openly discussed their past and present participation and it was possible to build a picture of their involvement over time and assess how well survey responses estimated involvement.

    An examination of survey responses coupled with an analysis of qualitative material reveals a picture which is not as straightforward as respondents perhaps thought. As already mentioned the first South Yorkshire question on involvement in local organisations did not pick up membership of local social clubs or working men’s clubs. Given respondents understanding of the question and the overall level of involvement described this is not really surprising. However, in five cases where respondents had answered ‘no’ to the local organisation question their social club membership was not subsequently picked up by the second South Yorkshire question, even though this option was included in the list of groups.

    With regard to the second South Yorkshire question, it seems likely the 2004 survey responses under reported the extent of involvement in groups. Table 1 shows the number of groups reported in the survey compared to the estimated number our additional assessment suggests was the case in 2004.

    Table 1: Number of Groups Reported

    Number of times chosen in 2004 South Yorkshire Survey

    Qualitative interview assessment of position at 2004

    Hobbies/social clubs 5 14
    Sports/exercise 5 8
    Adult education 4 7
    Local community/neighbourhood 5 5
    Children/young people 5 5
    Trade Union 1 5
    Religious 4 4
    Older people 2 4
    Health, disability, welfare 2 3
    Political 1 2
    Environmental 1 1
    Other 1 1

    The category with the largest discrepancy is hobbies and social clubs followed by trade unions but some of the groups only show a small difference or none at all. It is worth mentioning the under reporting of trade unions. Subscription to a trade union alone would not be counted in the GHS 2004/5 questions. Instructions for interviewers indicate that simply paying a subscription or being a member of a group but not interacting with other members does not qualify (Green and Fletcher, 2003). The extra South Yorkshire cases fall into this category whereas the one case which the 2004 survey originally picked up was extremely active in trade unions.

    Of 36 respondents, our analysis overall suggests the survey responses for 19 were accurate and 12 were inaccurate. In five cases it was not possible to make an assessment since information was either unclear or missing from the survey. Of the 12 incorrect cases, four did not pick out any groups and eight were involved in other activities which were not picked up by the second question. Of these eight, half did not mention trade union membership.

    Conclusions and Concerns for Researchers and Policy Makers

    The evidence presented discusses some difficulties associated with measuring volunteering and participation generally. It is widely recognised that measuring these subjects is not straightforward and the methodology employed in surveys plays a part in influencing the levels of involvement reported. As a result cross survey comparisons can be problematic. Interpreting data from different surveys and sources needs an awareness and understanding of how different approaches and variations in questioning techniques may influence responses.

    Researchers embarking on work in this field at a local level need to carefully consider the robustness of the questioning methods employed to measure participation, and how questions from larger scale studies are incorporated into local surveys. Our comparison of questions measuring levels of participation in groups and organisations in the GHS and adapted in the South Yorkshire Survey, indicates how even small differences in the questioning methods and wording employed influences the data collected and affects data comparability. Our qualitative interviews suggest there may be under reporting of groups in some categories.

    In addition evidence from qualitative interviews emphasises the importance of considering respondents understanding of the questions and the context of local communities in which surveys are being undertaken. Whilst our findings may not be surprising given the legacy of close knit mining communities in the study areas, they do illustrate a particular issue about asking questions which relate to a local area and whether this area should be defined for respondents or not. The Place Survey and the Citizenship Surveys, for example, ask respondents to consider their local area to be the area within 15 to 20 minutes walking distance from their home. Such instructions will be designed to standardise responses, increase comparability and to aid comparison across other surveys which use similar definitions, but it is possible they impose a definition of ‘local’ on respondents which may not correspond to what they regard as their local area.

    The UK Government’s success in promoting volunteering and participation is often expressed in terms of the level of volunteering and active participation citizens are involved in. As such the methodological concerns related to the measurement of participation and volunteering are important since they have a bearing on how progress towards a key policy target is interpreted. A recent briefing by nfpSynergy criticises the Government’s definitions of volunteering and civic participation employed in the Citizenship Survey and argues that they over estimate levels of volunteering (Saxton and Baker, 2009). Whether their claims are well founded or not, debate around the boundaries of what constitutes volunteering raises questions about what is actually being measured and muddies the water when it comes to assessing how effective investment aimed at increasing levels of volunteering may be.

    Locally, the lack of reliable data on volunteering makes it difficult for policy makers and other local partners to plan strategically, to monitor progress towards targets and to assess the impact of policies aimed at developing volunteering. Whilst the Place Survey will help to standardise data collection across local authorities and will help to provide comparable data on volunteering at a local level, there are problems associated with it. The data it collects will not be directly comparable to those provided by the Citizenship Survey. The Place Survey employs a much shorter question which is likely to result in lower levels of reported volunteering than those in the Citizenship Survey. It will be difficult to accurately benchmark local performance on volunteering targets in LAAs against the national and regional picture. Baselines for performance targets in LAAs will also be set by the first wave of the Place Survey. It is likely that targets concerned with volunteering will be based on data which do not capture the full extent of participation and volunteering which is already taking place locally.

    Notes

    1 When the data were initially analysed there was an apparent rise in reported levels of involvement in local organisations between 2000 and 2004 in the South Yorkshire Survey.  Whilst it was possible that these figures indicated a substantial increase in participation levels in the communities, such an increase certainly needed treating with caution as there was the possibility that the addition of a second question in 2004 could have influenced responses.  However, further exploration of the data suggest that the rise could largely be accounted for by the way the data have been cleaned.  If respondents had answered ‘no’ to the question on local organisations, but had picked out a group from the second question, then their response to the first question had been recoded to a ‘yes’.  The effect was to considerably inflate the rise in the level of involvement in local organisations.  The actual increase was from 21 per cent to 23 per cent for main sample and from 27 per cent to 31 per cent for the longitudinal sample.

    * Correspondence Address: Jan Gilbertson, Centre for Regional Economic and Social Research, Sheffield Hallam University, Unit 10, Science Park, Howard Street, Sheffield, S1 1WB.  Email:j.m.gilbertson@shu.ac.uk.

    References

    Attwood, C., Singh, G., Prime, D., Creasey, R. and others (2003) 2001 Home Office Citizenship Survey: People, Families and Communities.  London: Home Office.

    Blaxter, M., Poland, F. and Curran, M. (2001) Measuring Social Capital: Qualitative Study of How Older People Relate Social Capital to Health.  London: Health Development Agency.

    Brannan, T., John, P. and Stoker, G. (2006) Active Citizenship and Effective Public Services and Programmes: How Can We Know What Really Works?  Urban Studies, 43, 5/6, 993-1008. CrossRef link

    Communities and Local Government (2006) Strong and Prosperous Communities.  The Local Government White Paper.  London: Communities and Local Government, Crown Copyright.

    Communities and Local Government (2008) National Indicators for Local Authorities and Local Authority Partnerships: Handbook of Definitions.  London: Communities and Local Government, Crown Copyright.

    Cote, S. and Healy, T. (2001) The Well Being of Nations: The Role of Human and Social Capital.  Paris: OECD.

    Davis Smith, J. (1998) The 1997 National Survey of Volunteering.  London: Institute for Volunteering.

    Evans, E. and Saxton, J. (2005) The 21st Century volunteer: A report on the changing face of volunteering in the 21st Century, [online], last accessed on 7th June 2006 at www.nfpsynergy.net.

    Field, J. and Hedges, B. (1984) A National Survey of Volunteering.  London: Social and Community Planning Research.

    Gilbertson, J., Green, G., Grimsley, M. and Manning, J. (2005) The Dynamic of Social Capital, Health and Economy.  The Impact of Regeneration in South Yorkshire Coalfield Communities.  Sheffield: Sheffield Hallam University, Centre for Regional Economic and Social Research.

    Gilbertson, J. and Manning, J. (2006a) The Dynamic of Social Capital, Health and Economy.  Volunteering Social Capital and Civic Engagement in South Yorkshire Coalfield Communities.  Sheffield: Sheffield Hallam University, Centre for Regional Economic and Social Research.

    Gilbertson, J. and Manning, J. (2006b) Social Participation and Social Capital in South Yorkshire Coalfield Communities.  Voluntary Action, 8, 1, 22-38.

    Green, G., Grimsley, M. and Suokas, A. (2000) Social Capital, Health and Economy in South Yorkshire Coalfield Communities.  Sheffield: Sheffield Hallam University, Centre for Regional Economic and Social Research.

    Green, H., Connolly, H. and Farmer, C. (2004) 2003 Home Office Citizenship Survey: People, Families and Communities.  London: Research and Statistical Directorate, Home Office Crown Copyright.

    Green, H. and Fletcher, L. (2003) Social Capital Harmonised Question Set.  A guide to questions for use in the measurement of social capital.  London: Office for National Statistics.

    Hall, M. (2001) Measurement Issues in Surveys of Giving and Volunteering and Strategies Applied in the Design of Canada’s National Survey of Giving, Volunteering and Participation.  Nonprofit and Voluntary Sector Quarterly, 30, 3, 515-526. CrossRef link

    Home Office (2005) The Volunteering Code of Good Practice.  London: Home Office.

    Home Office (2006) New Boost for Volunteering.  London: Home Office (www.gnn.gov.uk)

    Kitchen, S., Michaelson, J., Wood, N. and John, P. (2006) 2005 Citizenship Survey Active Communities Topic Report.  London: DCLG.  Crown Copyright.

    Lowdnes, V., Pratchett, L. and Stoker, G. (2001) Trends in Public Participation: Part 1 – Local Government Perspectives, Public Administration, 79, 1, 205-222. CrossRef link

    Low, N., Butt, S., Ellis Paine, A., Davis Smith, J. (2007) Helping Out: A national survey of volunteering and charitable giving.  London: Office for the Third Sector, Cabinet Office.

    Lyons, M., Wijkstrom, P. and Clary, G. (1998) Comparative studies of volunteering: What is being studied?  Voluntary Action, 1, 1, 45-54.

    Lynn, P. and Davis Smith, J. (1991) The 1991 National Survey of Voluntary Activity in the UK.  Berkhamsted: Volunteer Centre UK.

    Lynn, P. (1997) Measuring Voluntary Activity.  Non-Profit Studies, 1, 2, 1-11.

    O’Neill, M. (2001) Research on Giving and Volunteering: Methodology Considerations.  Nonprofit and Voluntary Sector Quarterly, 30, 3, 505-514. CrossRef link

    Office for National Statistics (2006) Social and Vital Statistics Division.  General Household Survey, 2004-2005 [ghs04soccap_archive.sav].  Colchester, Essex: UK Data Archive [distributor].  SN: 5346.  Last accessed 20/05/2009

    Office for National Statistics (2008) Harmonised Concepts and Questions for Social Data Sources Secondary Standards: Social Capital.  London: ONS, Crown Copyright.

    Rooney, P., Steinberg, K. and Schervish, P.G. (2004) Methodology is Destiny: The Effect of Survey Prompts on Reported Levels of Giving and Volunteering.  Nonprofit and Voluntary Sector Quarterly, 33, 4, 628-654. CrossRef link

    Saxton, J. and Baker, J. (2009) How Government Definitions Over-estimate Levels of Volunteering.  London: nfpSynergy.

    Steinberg, K.S., Rooney, P.M. and Chin, W. (2002) Measurement of Volunteering: A Methodological Study Using Indiana as a Test Case.  Nonprofit and Voluntary Sector Quarterly, 31, 4, 484-501. CrossRef link

    Williams, C. (2003) Developing Community Involvement: Contrasting Local and Regional Participatory Cultures in Britain and their Implications for Policy.  Regional Studies, 37, 5, 531-541. CrossRef link