Not Another Survey! Conducting the APCOL Questionnaire in KGO

by Joseph E. Sawan

On the heels of two APCOL case studies; the housing case study in Kingston Galloway – Orton Park (KGO) and the food security community leadership development case study with FoodShare the APCOL survey began with the support and direction of a team of animators and organizers who have led antipoverty campaigns in their communities. After a year of survey committee meetings, the survey was finalized and we were ready to conduct our first interviewer training. Rather than rely solely on graduate student researchers, it was clear that our plans to incorporate the energy coming from the case studies could help organize and design a unique approach to survey research.

From our experience with the case studies, it was clear that participants would eagerly take on the research component of antipoverty activity, and that the broader hopes for a truly collaborative university/community research approach could be realized. It was not without contradictions and it took enormous efforts on both sides. But, these efforts are bearing important new fruit in terms of research process and outcomes. In this reflection, I outline the learning that took place, challenges faced in participatory research and some positive outcomes that are taking place as a result of our method.

From Case Study to Survey

For my part, the roles of case study coordinator and survey researcher became heavily intertwined. For both KGO and FoodShare case study participants, there was a great deal of interest and excitement to continue some level of involvement with APCOL beyond the case studies. We returned to the animators and organizers with our plans for a broad based survey in eight GTA neighbourhoods. Most of the participants were eager to join us and engage in survey research, and several of them had previous experience in community research.

Crucial to the success was our two-day training, led by D’Arcy Martin in a lively and participatory fashion. Participants in both APCOL case studies came together to learn the ins-and-outs of the APCOL questionnaire, as well as shape and confirm our survey approach. The level of engagement among the participants increased throughout the training and by the end everyone reached a new level of commitment to the project and antipoverty activity in their respective communities.

At this stage, participants faced a shift in their roles with APCOL, and it did not come free of challenges. Working through scheduling and workplans, the time commitment facing the researchers is significant and the honorarium for researchers does not cover the level of diligence and skill required of them. For our work to be successful we had to respect existing commitments and develop realistic workplans that would not stress or alienate them in their community involvement. One solution would be to provide continuity with the case studies and overlap with their existing community activities.

The East Scarborough Festival Market

As a part of the KGO case study, participants engaged in tabling at the Festival Market organized by the East Scarborough Storefront during the months of June and July. They conducted informal interviews to better understand the various housing issues facing residents in the community as a part of the campaign to develop a local affordable housing strategy. This experience provided the organizers with the knowledge and confidence to administer and understand the potentially powerful role of the APCOL questionnaire in supporting organizing.

The organizers and animators now put on their "researcher hats" and returned to the Market with a new twist to their organizing strategy. With four graduate students and four community researchers on our first day, we managed to complete seven interviews during the four hours there. Most people we approached were receptive to the project and had time to participate. Since the KGO researchers are well-known in the neighbourhood, we had very few issues finding participants.

For the first day, we paired up community researchers and graduate students in order to encourage learning from each other about interview techniques and navigating the questionnaire. This proved extremely helpful as it brought to our attention specific issues with the questionnaire that could be addressed immediately.

In one instance, an interviewer noted that the respondent is quite involved in anti-poverty activities, but in her response to the questionnaire claimed to not be involved. Here we came face-to-face with a term – community activist – that was far more complicated than either we or the research literature had previously imagined. Practically speaking, we realized that the manner in which interviewers were asking the preliminary questions needed to be further clarified in order to improve the quality of the answers. Also, definitions of phrases such as "anti-poverty campaigns or activities" were necessary to help frame what types of activities are included. Often respondents would feel trapped by such specific terms, and the ability of the interviewer to provide an explanation determined the quality of the data.

The following week, community researchers showed significant development in their command of the questionnaire. Once again, several student researchers joined the team, but this time most of us conducted interviews alone. Following completion of each interview, the researcher would return to the APCOL table and would debrief with myself or another researcher while reviewing the questionnaire. This process allowed the opportunity to reflect on challenges encountered and strategies for improving the interview process. The informal aspect of this learning process is important to highlight.

After the first two weeks, researchers began to conduct interviews on their own with neighbours, family and colleagues. Once we completed around 30 interviews, we evaluated the demographic data of interviewees to determine if the breakdown was representative of the neighbourhood. Noting that the majority of respondents were past participants, non participants, women and over 25, we realized that to reach a somewhat representative sample would need to target more male, current participants between 18 and 25. At this point researchers used their networks to reach out to male, current participants and young adults. It proved very effective, and researchers worked together to identify potential participants.

Not Another Survey!

As one of the GTA’s priority neighbourhoods, residents in KGO are no strangers to academic and community-based surveys, and one of the most significant challenges that interviewers faced was ambivalence towards the interview. Community researchers were clear about these issues during the training and provided concrete solutions to improve the administration of the survey.

First, it was necessary to refer to the survey as a questionnaire and interview as a means to better describe the depth and unique characteristics of the overall APCOL project. Like the word ‘activist’, the word ‘survey’ was also pre-loaded with a specific meaning for many.

Second, rather than approaching the questionnaire as strictly data collection, researchers learned to conduct the interviews in a professional and scientific manner while building new networks with interviewees who showed interest in the local campaign.

Such an unorthodox approach to quantitative research has so far proved to have significant advantages in the quality of data, as well as opportunities for building stronger community-university relations. At the same time, there are inherent challenges as well that we mustn’t ignore, but in our first completed neighbourhood, it is clear that the advantages far outweigh potential challenges in the data analysis.

Furthermore, there is strong evidence that the mixed methods approach to research provides us with unique opportunities to understand dynamics in a more holistic manner. By implementing a participatory approach throughout the research, in as many aspects as possible, academics and community partners are able to contribute to each other’s efforts in ways that encourage innovative strategies for social change.

Next Steps

As we move forward to complete the survey across the GTA, as field experts the team of community researchers will now have the opportunity to help train and mentor the next team of community researchers. Equally important, the community researchers will also join graduate students in the data analysis process to help us dig into the immense amount of information collected on the dynamics of anti-poverty activities in KGO. As we begin the process together, we will develop a unique understanding of anti-poverty activity throughout the GTA.

All of us encountered a great deal of learning throughout the first phase of the APCOL questionnaire, and the lessons that we are applying to subsequent neighbourhoods will continue to bring us closer to a truly participatory process. Opportunities to reflect on the process each step of the way have tremendous value to ensure that we are true to the goals of the APCOL project. In order to adhere to the values of participatory research, we must continue to struggle with the challenges and contradictions at every level of research and community activity.

Joseph Sawan is a graduate student in Sociology and Equity Studies in Education at the Ontario Institute for Studies in Education, University of Toronto. He completed his M.A. in Adult Education at San Francisco State University and has a background in union and community organizing in California. Joseph’s current research is on the dynamics within social movements and the learning processes involved.