Formative Evaluation: New York at Its Core

The Museum of the City of New York (MCNY) contracted Randi Korn & Associates, Inc. (RK&A) to conduct a formative evaluation of proposed exhibits for “The Future of the City Lab,” an interactive gallery to be featured in the New York At Its Core exhibition that invites visitors to think about and help design the future of New York City.

How did we approach this study?
The goal of the study was to explore visitors’ opinions and understandings of five specific exhibits—the Introduction Text, the Big Challenges Wall, a Neighborhood Exploration, the “Density” iPad Activity, and the Selfie Station—to inform development of the “Future of the City Lab.” RK&A conducted open-ended, in-depth interviews with a total of 44 visitors representing 3 groups (7 NYC schoolteachers, 20 millennials, and 17 walk-in visitors). To begin each interview, participants looked at mock-ups of the proposed exhibits. The mock-ups consisted of small-scale reproductions of several text and graphic panels, slideshows simulating the games and activities, and a list of possible Selfie Station prompts. RK&A then spent 25-40 minutes guiding participants through the mockups and interviewing them about their perceptions and opinions of the various presentation strategies, main messages, and future placement in “The Future of the City Lab.”

What did we learn?
In general, the proposed exhibits for “The Future of the City Lab” tested well. The content resonated with participants on several levels—for example, some local residents related to struggles of affordable housing; others, who work in urban planning, had a professional understanding of the transportation issues; some foreign tourists live in cities facing similar challenges; and, other tourists live in small towns but have a deep curiosity about New York City. Nevertheless, some barriers to interest and understanding emerged. For example, some participants felt “The Future of the City Lab” emphasizes challenges over solutions, to a fault, and others felt overwhelmed by the data visualizations. Teachers pointed out challenges that may be specific to school children; for instance, a few said the content may be uncomfortable for some students since many of them represent New York City populations that struggle the most with the Big Challenges (like poverty). Finally, participants across groups expressed concerns about sharing their exhibit contributions (like building designs or selfies) with other visitors or through social media.

What are the implications of the findings?
Findings demonstrate visitors are extremely interested in the concepts presented in the “The Future of the City Lab,” but interpretation may need further articulation and clarification to help them understand the main messages and the connections between the activities. As MCNY staff continue to develop “The Future of the City Lab” and New York At Its Core, they might consider ways to emphasize proposed “solutions” alongside challenges; perhaps by including photographs of architects’ designs for future buildings in iPad activities. Simplifying or reducing the number of data visualizations in the Big Challenges, and continuing to test them to ensure visitors understand their messages, will also help visitors feel less overwhelmed. And, providing ways to contribute content without also sharing personal data (e.g., building designs but no selfies) will further support visitors’ interest in helping to design the future of the city.