Skip to main content

Questions (and Answers) as Data Mindset Takes Hold

It’s all about the questions. As neighborhood groups build databases and dig through the numbers to see what they’ve got, the one thing that keeps coming up in abundance is questions. “What exactly are we doing?” “How well are we doing it?” “Is it making a difference?”

And then the toughest one: “Are we asking the right questions?”

Collaboration around safety and youth in Little Village led to formation of a new group, Padres Angeles (Parent Angels), which is organizing to reduce violence of all kinds.

Gordon Walek

For the last 18 months, five neighborhoods have been tackling local issues through data-informed programs that were designed to “Test The Model” of focusing multiple community-based partners around tightly defined interventions.

The projects are diverse and weighty, ranging from education in Logan Square and Pilsen to foreclosure recovery in Chicago Lawn. A dozen groups in Little Village are pooling expertise around at-risk youth, and in Humboldt Park an equal number have taken on the obesity challenge by pushing fitness, healthier eating and lifestyle changes. In each case, the projects build on 10 years of relationship-building and trust created through LISC’s New Communities Program (NCP).

What’s common among all the neighborhoods is a data mindset, which was a requirement for participation in Testing The Model (TTM), a three-year experiment managed by LISC Chicago and funded in large part by the MacArthur Foundation.

“All of us need to be better able to talk about our work and whether it is having an impact,” says Taryn Roch, LISC Chicago’s program officer for evaluation and impact. “But data for data’s sake is useless,” she adds. “You need to have a question that you want answers to.”

Ramping into data

The groups – and LISC Chicago itself – have come a long way. A 2011 assessment of the technical and data capabilities of LISC’s lead agencies, conducted by Roch during her previous work with the Metro Chicago Information Center, found that most groups lacked the time, expertise and resources to use data in meaningful ways. Many had antiquated hardware and software, lacked staff with technical skills, and didn’t have the funding to build capacity.

TTM is changing that. The MacArthur investment allowed each neighborhood to develop a “theory of development” around a specific issue and a series of interventions designed to make a difference over time. The structured approach advanced the previous NCP methodology by emphasizing evidence-based practices and use of data to inform design, improve programs and track progress. 

Logan Square’s TTM project is tracking participation of parents in community campaigns, including a voter registration training in late 2013.

Gordon Walek

By providing funding to build databases and hire part-time data coordinators, TTM has allowed all five groups to dig deep into the world of data. See summaries of all five programs. It’s been a bumpy, drawn-out process, with a fair share of tooth-grinding and furrowed brows. But as groups learn how to accurately collect and analyze data, they are recognizing how valuable it can be for program improvement, reports to the boss, and documentation for funders.

“Everything for so long was just on paper – we had boxes of it – so seeing it all come together in one place has been very useful,” says Suheily De Olio, the data coordinator for Logan Square Neighborhood Association (LSNA). That project hopes to measure the impact that LSNA’s acclaimed parent-mentor program has on the parents themselves as well as the children in the classrooms.

Those high-level answers aren’t yet available, because it took considerable time and thinking to develop adequate survey and observation methodologies. But the first waves of data are already proving valuable. Instead of the paper sign-in sheets that previously tracked participation, the database now provides information on 526 parents, including what meetings each has attended and what roles they played in community advocacy campaigns – everything from room prep or minor speaker to chair of the meeting. It also tracks which community campaigns each has been involved in (such as immigration or education). “We’re able to see how individuals are developing as parent leaders,” says De Oleo. “And other organizers at LSNA are asking to use the database to find parents who are active on different issues.”

Humboldt Park puppet shows about nutrition bring double benefits: reaching young children and families at child-care centers, and giving performance opportunities to young puppeteers trained by La Casa Norte. (link: http://www.lisc-chicago.org/news/2470)

“We’re learning an enormous amount,” adds Kathryn Bocanegra, who manages the youth-focused interventions in Little Village. That neighborhood’s 12 TTM partners are using detailed pre- and post-surveys of participating youth to better understand the pressures they face, and to measure the impact of youth programs ranging from boxing and urban gardening to mental health counseling and conflict-resolution training.

“I’ve seen the light bulb go off in different people’s heads,” says Bocanegra about the early data-collection efforts. “TTM is helping them develop the language and tools to articulate their program’s impact.”

Garbage in, garbage out

Unfortunately, every one of the five groups has learned the age-old lesson about “garbage in, garbage out.” Some of the early survey data in Little Village, for instance, was compromised because of inadequate training in how to administer the surveys, which resulted in some youth filling out the forms themselves instead of doing it with a trusted adult. In Humboldt Park, where surveys were designed to measure changes in attitudes and behavior related to fitness and nutrition, the first round captured plenty of pre-program data, but many post-program surveys were not completed.

And it’s often difficult to find meaning in the rivers of data. “We asked around the table at a meeting and most of the program partners said the data just wasn’t that compelling to them,” said Sandra Gray, associate director of Humboldt Park’s lead agency, Bickerdike Redevelopment Corporation. “They can see that people’s perceptions are changing, but it doesn’t come naturally how to use this data” to prove that changes have taken place.

One thing the data did make clear was that most of the programs – which ranged from zumba dance classes to nutrition education to a reading club – faced high attrition rates. Simple attendance data showed that participation fell off significantly from initial enrollment through the end of each program. “So the groups are all working on that, and sharing ideas about how to keep people coming back,” said Aracely Galvan, the data coordinator in Humboldt Park.

“This is exactly the type of learning that we were hoping for,” said Alaina Harkness, program officer for community and economic development at the MacArthur Foundation. “Figuring out how to really use data to improve impact is vital to the community development field, and LISC and its partner groups are well-positioned to do that.”

Logic first, then program

None of this would have been possible without some heavy lifting by various technical and subject-area consultants that helped each group first formulate appropriate research questions and then design data systems to capture the right information. Helping out were the DePaul University Institute for Housing Studies, which provided baseline data and housing expertise for Chicago Lawn’s effort to repopulate vacant homes; the Consortium to Lower Obesity in Chicago Children (CLOCC), which designed the surveys used in Humboldt Park; and the Adler School for Professional Psychology, which supported Little Village’s youth network. Chapin Hall at the University of Chicago had a dual role, advising on the education projects while fielding a separate team to build five custom databases and train local groups on their use. Another partner was James Connell, president of the Institute for Research and Reform in Education, who worked with the groups to tighten their theories and related interventions.

Once they got the logic right and the program resources lined up, all groups began implementation in 2013. In Pilsen, The Resurrection Project partnered with Chicago Public Schools and an early childhood development provider to extend pre-school to a full day at one local school, and expanded its support to local middle school students to improve their transition to high school. In Chicago Lawn, the Southwest Organizing Project gained a $3 million grant from the Illinois Attorney General’s office to begin buying and rehabbing vacant homes. It built a database of hundreds of potential homebuyers and renters, and is now deploying a mobile app from LocalData to map vacant properties in the target area.

A dozen Little Village organizations are coordinating their efforts to build stronger support networks for youth. Above, a peace march in late 2013.

In Little Village, as the various agencies better understood the needs of local youth, they have expanded their collaboration through joint programming and referrals, fulfilling one of the early goals of the program. 

A larger overall goal of TTM is to test whether multiple partners can organize around a common agenda to create “collective impact.” LISC’s programs for years have incorporated the key drivers of collective impact but lacked the shared agenda and data tracking that are central to TTM. It’s still early; the projects haven’t yet produced enough data to prove impact. But one thing is clear. The community teams are asking a lot of the right questions, and developing the kinds of data that will help them find answers.

For more information, contact Taryn Roch, troch@lisc.org, 312-422-9554.

Posted in General news

STAY CONNECTED

Stay up to date with the the latest news and events related to LISC Chicago.

Facebook
Twitter
YouTube
Flickr
Instagram

About LISC Chicago

Local Initiatives Support Corporation Chicago connects neighborhoods to the resources they need to become stronger and healthier.

More about LISC Chicago »
Contact our staff »