What does good shareholder engagement look like?

On Friday night, shareholders began receiving emails from 3x3, the research firm our Board hired to do shareholder engagement. 

We strongly encourage every household to complete the survey, and to do it ASAP so you are eligible to participate in the “listening sessions.” If you don’t see the survey in your inbox—which is apparently the case for a lot of shareholders—we’ll explain what to do later in this post. 

Now, onto some thoughts on the survey and the roll-out!

Jump to a section:

Since the Board announced they would be working with 3x3, a lot of people have been asking us whether we think they are a legitimate research firm. The answer is: Absolutely! In fact, 3x3 is one of the firms that Guido personally recommended that the Board hire for this. And a third party research firm should be able to help our Board conduct a rigorous and robust survey, so long as the Board does not impede that firm’s ability to engage shareholders. It won’t work if the Board has a predetermined solution in mind and directs the researchers to gather data that supports their desired outcome. 

We have also heard some frustration from shareholders related to the fact that the Board is now spending even more money to do shareholder engagement, after spending $2.5M on designs and other work that is now changing. If our leadership had simply done this from the beginning instead of trying to bypass this step, the thinking goes, couldn’t we have avoided wasting money? 

These are completely valid concerns, and we agree that this could have—and should have!—been done differently from the start. But instead of letting sunk cost fallacy send us further down the wrong path, we ultimately think it’s better to change course and do right by shareholders. We are not dismissing those sunk costs, but we still believe that doing shareholder engagement and moving forward with a plan that is rooted in data and research, and that reflects the desires of a majority of shareholders, is the financially prudent thing to do, and will save us money in the long run. And hiring a firm with expertise doing this exact kind of engagement really is the best way to plan a lobby renovation that people here feel really good about, which is ultimately what we hope to see happen. 

Of course, if the Board exerts pressure on 3x3 to try to push through their preferred elements of the design, shareholders could end up spending more money and still not getting a design that responds to the community’s needs. Our Board has a history of hiring legitimate firms with expertise in research best practices, doing high-quality shareholder engagement, and then simply burying or downplaying the results. The only way bringing on 3x3 will be a poor use of funds is if the Board repeats the same mistakes made in the past when it comes to community engagement, or if the Board gets in their way when it comes to execution.  

Here’s the timeline for shareholder engagement, based on what 3x3 put in their email/on the survey landing page: 

Friday, May 1 at 6:30 p.m.: Two emails are sent to shareholders about the new survey; one email includes a link to take it. Not all shareholders receive these emails. 

Note: If you want to be eligible to enter the lottery for one of two “listening sessions,” you will need to complete the survey by either Thursday, May 7 at 9:00 a.m. or Thursday, May 8 at 9:00 a.m., depending on which communication you reference. This is confusing—do they mean Thursday, May 7, or Friday, May 8?

In any case, this is a very quick turnaround. It means that anyone who fills out the survey after May 10th has no chance of being included in a listening session. And if you haven’t even received the survey yet, you’re really not left with a ton of time to complete it. 

Only 20 shareholders will be selected for each “listening session.”

Monday, May 4, 5-7 p.m.: 3x3 will be at Seward Park Co-op tabling to promote the survey and answer questions about it. They say they will also offer an opportunity for shareholders to get help filling out their survey, and it seems like shareholders can pick up a paper survey here too. (We believe you will also be able to get a paper survey from the management office but have not confirmed with them yet.) 

Thursday, May 7, 9 a.m.: The apparent deadline for completing the survey if you want to be included in the lottery for a listening session. 

Tuesday, May 12, 12–1:30 p.m.: First of the two listening sessions (this one is virtual), limited to 20 people. 

Wednesday, May 13, 6-7:30 p.m.: Second of the two listening sessions (this one is in person), limited to 20 people. 

Sunday, May 17, 3-5 p.m.: 3x3 will be here tabling again. Shareholders can get help filling out the survey during this time, but it will be too late to be considered for a listening session at this point. 

Friday, May 22: The survey closes.

Summer 2026: A summary of survey results will be shared with shareholders. 

Overall, we’re looking at an extremely tight turnaround for both the survey and listening sessions that is out of step with most research projects of this nature; we suspect everything is being expedited so that it is completed before Board elections in June. Waiting this long to take action and then cramming everything into three weeks will likely depress the number of responses and lower the quality of the process overall. 

The timeline is also a problem because, as a general rule, researchers need a lot of time to recruit the right mix of participants for “listening sessions” if they want to do them right. 

To help everyone better understand what this process should look like, we spoke with a shareholder with a professional background in user research. Here’s how they explained it.  

Research methods typically fall into one of two categories: Qualitative and quantitative. Qualitative research involves talking to a small group of people that are representative of your larger population in order to understand peoples’ general attitudes, see how they think about different problems, and glean insights into how they interact with a space. When we say “talk to,” we don’t mean “have a casual conversation.” We mean employing legitimate research methods like interviews, focus groups, diary studies, etc. Ultimately, researchers should be using this time to ask a relatively small but representative number of people structured, non-leading questions. 

Qualitative research is generally done first because it’s meant to give insight into how or why people do the things they do; those insights are then used to inform quantitative methods—like a survey—which can tell you the prevalence of desires across your population. You do it this way so that the multiple-choice questions cover most of the things people might have opinions on, and aren't just based on your own preconceived notions. 

Now, of course, researchers can’t interview all 5,000 residents here, but they can talk to people who generally match the demographic profile of those 5,000 residents and offer a good representation of the whole. For example, let’s say we’re going to conduct a focus group with 20 residents, and 20% of all Seward Park residents have mobility issues. In that case, 4 of the 20 participants should probably be people with mobility issues.

3x3’s survey does include some “screening” questions. For example:

There are a couple of issues with the “screening” approach being used by 3x3, though. 

First, the order of operations here is a bit odd. Again, best practice is to do qualitative research first to inform how you’ll write the survey. It’s certainly possible this new survey was based on the 2018 qualitative and quantitative research done around the air rights, and they are using that as a stand-in for new qualitative research; this approach can certainly work when the original research is high-quality and remains relevant. But regardless, doing both at the same time (while also limiting listening session participation to people who complete the survey in the first five days) is fairly unusual, and, again, implies that 3x3 was given an expedited timeline to work on. 

Next, unless the Board has an extensive demographic breakdown of our co-op (that is, they have a way of knowing how many households in total have one or more children under age 7), there’s no way to represent that number within the listening sessions—these results will simply just tell us the percentage of survey respondents who have these qualities. So, as an example, if 70% of respondents indicate that they’ve lived in Seward Park for less than 5 years, will 14 of 20 listening session participants be individuals who have lived here for less than 5 years? We can tell at a glance that this isn’t an accurate snapshot of our co-op, but we simply have no idea what is. 

All of this is compounded by the tight timeline; to even be eligible for the listening session “lottery,” you only have a few days to respond to the survey. Given that, and given the significant tech issues thus far and the lack of paper communications, it seems likely that the early respondents will be more tech-savvy, which could translate to people who are younger. To recruit a good, representative mix of participants, you just need time, which is not what we have right now, and which will likely undermine the results. 

It’s also unclear why survey responses are limited to one per household. The stated reason is “consistency in responses,” but what does that actually mean? Imagine a household that consists of one person who is in their 40s and able-bodied and one person who is in their 80s and who has mobility issues. Their responses would likely vary significantly! Even spouses might have very different desires when it comes to community spaces, amenities, and priorities. How does this household then negotiate what their responses to the survey questions should be? (More time for them to discuss and consider the options would likely help!)

Finally, it’s not clear why they limited the listening sessions to only 40 people total. If the participants were well-screened and closely aligned with our co-op’s demographics, this would likely be enough people to represent the larger population and inform a survey…but because that isn’t happening, we have concerns that this will be a truly representative sample.

Now, having said all of this: 3x3 is a reputable firm and it shows in the construction of the survey itself.

Research questions should be nondirected, meaning they don’t lead a person answering them to think there’s a “right” answer. Qualitative research is flexible in order to avoid cramping respondents’ answers, but survey questions need to be more precise and restricted in order to be unambiguous. Most survey questions should be closed ended. Think: Multiple choice questions, checklists, and statements followed by a range of options (strongly agree, strongly disagree, etc., with a neutral middle option). In order for closed-ended questions to be easily answerable, the available options need to be specific, exhaustive, and mutually exclusive. Generally speaking, this survey follows these best practices. 

Question 16: When thinking about capital improvements, which of the following should be prioritized? (Select up to 2.) 

This is a badly constructed research question because the options presented aren’t mutually exclusive, and because it mixes non-discretionary and discretionary spending. Beyond that, structural integrity, usability, and accessibility aren’t things that should be voted on; they should be inherent to anything we do, discretionary or otherwise. Also, if the building is crumbling, that is bad for quality of life and property values.

Question 17: Do you agree or disagree with the following statement:

"It is acceptable to move forward with improvements even when some shareholders strongly oppose them, if the majority supports the decision.”

Options: Strongly disagree, disagree, neither agree nor disagree (or neutral), agree, strongly agree

Good survey questions never shut people out, and always give them an option that they feel applies to them. In this instance, an "Other" choice with space for an open-ended response would be appropriate—it would allow you to say something like "yes, so long as the decisions don't cause accessibility issues" or "yes, so long as the decisions don't lead to higher maintenance fees."

We’re really not sure why this question is here; it feels like a pretty obvious trap, and raises concerns about how the Board thinks about representing our more vulnerable/underrepresented residents. For example, older people with mobility problems could be in the minority here, but we are also a NORC; if our older residents are concerned about safely moving through our spaces, should we simply disregard their views because a majority doesn’t have that problem? Good governance means balancing the desires of the majority with certain core values, and framing this question as a binary with zero nuance, context, or ability to expand on your answer is strange.  

Finally, some of the survey questions—even the well-written ones—clearly reflect what the Board wants to know about vs. what shareholders want to talk about. To an extent, this is normal: Surveys can’t go on endlessly—20 to 30 questions is perfectly reasonable for a survey!—so we wouldn’t expect this to ask about every possible amenity and design choice. What we’re left wondering is: What criteria was used to cull down survey questions to this final list? What did the Board direct 3x3 to focus on, and what existing information and data was used to inform that? 

Many people (us included!) report not receiving one or both of the emails; others say the emails are being sent to addresses they haven’t used in years. Here are the subject lines of the two emails that everyone should be looking for:

The sender is [email protected]

If you did not receive one or both of these emails, email 3x3 to let them know right away. Be sure to use the email address above; some official communications are directing people to email a slightly different one, which doesn’t appear to be working. 

Do NOT have someone else forward you the survey email, as each one has a unique link in it, specific to the recipient’s apartment number. If you use someone else’s link, they won’t be able to submit their answers or they will cancel yours out.

The fact that there are this many technical problems is a huge issue, and it’s made worse by the fact that people won’t necessarily know if they didn’t receive these emails. Why didn’t management send an email to everyone telling shareholders to keep an eye out for the email, informing everyone of the timeline (so they don’t delay in getting involved), and letting us know what to do if we didn’t receive it? 

We also know that a lot of people who live here simply aren’t tech-savvy/aren’t big email users, and some don’t use email at all. This survey is important enough that we believe the Board should have delivered a hard copy of the announcement and instructions for completing the survey to every unit’s door, with explicit instructions for how to get a printed paper survey. Our guess is that they didn’t do this because it would take longer, but the timeline here is fairly arbitrary. Taking another week or two to get a “once in a generation” renovation right seems perfectly reasonable. 

The flyers posted near the elevators also contain a lot of information in a fairly small font, organized in a way that makes it hard to read. 

Despite our concerns about this process—particularly the tight timeline, lack of good qualitative research, and chaotic communications—we still strongly encourage all shareholders to complete the survey. 

Participation is extremely important here! So please: Take the survey and then reach out to your neighbors to make sure they have taken the survey. We hope to see significant engagement to ensure the results represent the desires of as many shareholders as possible. The current process is creating meaningful obstacles to participation, so it’s going to take real effort to make sure it’s truly representative. 

We also strongly encourage the Board and 3x3 to do right by shareholders and extend this timeline. Between the serious technical issues and the fact that we have to compete in a “lottery” to attend a listening session—so there is no way for all shareholders to discuss different options openly, ask for more information/context when they need it, and share their thoughts freely—there are real reasons to worry that the results aren’t going to be representative of a variety of voices.  

Throughout this process, our main goal has been ensuring that shareholders have a voice: that means people who don’t use email or don’t have a friend on the Board should still be represented; that the process is transparent and the associated costs are known; that communications are clear and timely; and that the final results are both reflective of the majority viewpoint and mindful of the specific needs of minority populations here. It’s not too late to ensure that happens, but doing so will take real leadership and a sincere desire to make sure as many shareholders as possible are heard.