Below are a three short interviews captured by John Hale in preparation for the newsletter update. Interviews are with Urban Recipe Director of Food Co-ops, Spanga Gwabeni, Urban Recipe Executive Director, Jeremy Lewis, and Adam Lipus, the program evaluator.
“At the root of everything we do is community and so evaluating our work in a way that builds on the community and builds up the community just makes sense to us.” – Jeremy Lewis, Executive Director
Interview with Spanga Gwabeni, Urban Recipe’s Director of Food Co-ops
May 24th, 2018
John Hale: What are your feelings about the evaluation process and why is it important in your mind?
Spanga: I think that it is important for a number of reasons. Instead of assuming we’re achieving our goal of food security [for our co-op members], we will know for sure where we stand there. We are hearing from the people who are receiving the food instead of saying we’re creating food security based on the amount of food we distribute. We will also see and evaluate and find out what else we are giving our people besides food. The evaluation will confirm the things that we’ve heard from our members, and do so on a larger scale. We can find out what percentage of co-op members share their food with others, or feel a greater sense of self-esteem. The survey will also help [members] see where they were before they joined a co-op and where they are now, which I think will give them peace about knowing they don’t need to worry about food anymore.
John: What would you say are some of the things co-op members get out of being in a co-op in addition to the food?
Spanga: A sense of community. A sense of making a difference in other people’s lives by being with each other in times of need. Helping others raises people’s self-esteem. Going from not having enough food to being able to give other people food who are in need. It’s like even though I know I’m poor or “low-income” or “in need” I can still help others who are in the position I was before I joined the co-op.
John: How do the co-op members who are participating in the advisory committee feel about being involved in such a significant way?
Spanga: The members in the committee feel empowered. They haven’t been involved in this kind of process before. They feel like their voice is important and like they are important in the co-op, which is our goal. Overall, they are moved by the fact that they have been asked to participate in this process and they feel very empowered. They feel like it’s not just about the office making the decisions, that my voice is important.
Interview with Jeremy Lewis, Urban Recipe’s Executive Director
May 29th, 2018
John: Why is a more thorough evaluation of our food co-ops a priority at this time in the life of the organization?
Jeremy: Rightly so, our donors deserve to know the impact of their dollars, and as we look at expanding our work, it’s integrally important for us to understand that work and to build systems that provide opportunities to regularly evaluate and reflect and improve the model. We’ve got anecdotal stories we’ve heard from over the years and lots of good things to report, but how are we capturing that in a way that’s systematic and trying to learn from those insights, and involving the experts in the work of evaluation in helping us shape the model that we’re using?
John: Why does a participatory approach matter in an evaluation of this nature?
Jeremy: Our experts are the ones [co-op members] who’ve done it for 20 years. Or less even–if you’ve been a member for even 12 months you’ve developed some expertise in helping run the co-op. That’s why participatory evaluation is important–they’re evaluating and impacting the thing that’s going to have an impact on their own lives. Also, at the root of everything we do is community and so evaluating our work in a way that builds on that community and builds up the community just makes sense to us.”
John: Have you been pleased with the process and results so far?
Jeremy: I think the early part of the process has been great and I believe that we have a good plan in place and have assembled a good group of people to help us work that plan. Part of the challenge of community based work is managing the ebbs and flows of participation and continuing to find ways of encouraging people to own the work that they’re involved with. I was a little disappointed with the participation at our last evaluation committee meeting, but we all have to remember that the process is also the product.
Interview with Adam Lipus, Evaluator
May 24, 2018
John: What is the approach you’ve taken to this evaluation, how did you come upon the idea of postcards, and why do you think it makes sense and will be effective?
Adam: One of the core aspects of our approach is that it is a “participatory evaluation,” which basically means that we are emphasizing the inclusion of various stakeholders (board, staff, and co-op members) in evaluation discussions and decisions. This approach was ideal for two main reasons. 1) Participatory approaches are core to my professional practice, for ethical reasons as well as because participation increases evaluation quality. 2) Participatory evaluation is in line with Urban Recipe’s values of member dignity and ownership. The idea to do postcards came from this participatory process. During one of our early discussions with a group of co-op members, the members had a lukewarm response to the idea of doing surveys, which are a common evaluation method. But one member noted that surveys are fine if they can be short, like something that would fit on a postcard. The idea of doing a postcard format stuck, because a postcard is about checking in during a journey and giving a brief sense of what your life is like. In this case, the journey is each member’s individual journey from before the co-op to today. That forms the core of our before-and-after evaluation design, and the data are coming from a fun and simple postcard rather than a long survey.
John: How much have co-op members been involved in the process and why do you think it’s important to have their involvement?
Adam: In these discussions, members provided their insight on the benefits they have experienced through being in the co-op and how we might measure/document those benefits. Their input was really critical in informing our co-op “road map,” which lays out the intended outcomes of the co-op program. Now that we are in the process of carrying out the evaluation, we are engaging five co-op members as representatives on our evaluation advisory committee, where they are providing ideas/insight and will be serving as liaisons to other co-op members. Broadly speaking, involvement of program participants in discussion and decision-making about evaluation is really important, not only for ethical reasons but also because it can lead to better evaluation methods, more accurate data, and more relevant findings.
John: Have co-op members expressed anything positive or negative about being asked to be involved in the process?
Adam: Generally we have found members to be very engaged in participating in the evaluation process. Our focus group discussions were lively, with one member remarking that “I don’t think any of us really have a problem expressing how we feel about Urban Recipe.” And through the first two evaluation advisory committee meetings, all member representatives in attendance have been actively engaged in discussion.
John: Have the co-op members who are a part of the committee and those told about the plan in [co-op] meetings been affirming of the idea of the postcard survey?
Adam: Overall, the reception has been very positive so far. When we pilot tested the postcards with committee members, they remarked that the postcards were simple, easy, and “very explainable.” And one of them shared a sneak peek of the postcards with her peers during a co-op meeting and reported to me that the responses were all positive. From members’ perspectives, it is important that we are keeping the postcards anonymous, and some members have asked what we will be doing with the postcards. As an evaluator, I take the role of creating/collecting data very seriously. To bring this issue to the surface – I am a white male who is working with a group of co-op members who are predominantly black and female. In situations like these, I do my best to be mindful of my role and my interactions given the broader historical context of race and gender. As evaluators, our work can help programs learn and make the world a better place, but we have to be intentional in going about the process in a way that empowers and promotes collaboration rather than a way that perpetuates power imbalances.
John: How/why did you select the particular survey questions?
Adam: We designed the survey questions to measure specific outcomes in our co-op “road map.” Most of the survey questions were based on existing, validated questionnaires. We made some modifications for simplicity and to suit the context of the co-ops.
Thank you for reading. Please don’t hesitate to be in touch.