chapter one
GOALS DEFINE A CLEAR SURVEY PURPOSE

Conducting an effective employee survey requires a substantial amount of time, energy, and resources. You have to have a clear purpose for the survey, and the questions need to be worded accurately. You should minimize survey length to yield a response rate that is sufficient for scientific accuracy. The results should be presented in a format that maximizes usability, and you need to engage all of the relevant stakeholders in the feedback and action-taking process. These and more principles hold for all surveys regardless of length—even short “pulse” surveys. Each chapter of this book addresses one or more of these aspects. In this chapter we start with purpose.

Recognize the objectives and tradeoffs. A theme that runs through this book is that there are tradeoffs in employee survey design. You can’t have a survey that does everything for everyone while being short enough to elicit high response rates, so you have to choose one primary purpose—two at the most—and stay true to the purpose when deciding what to include and exclude. This means selecting the desired outcomes for the survey and the right organizational level or levels on which to focus. It also means recognizing the limitations of surveys.

Surveys are good for gathering information in a focused way from a large group of people. A survey can collect a lot of data quickly and cheaply, but it might increase decision-making time. Stakeholder interviews alone may identify the organizational issues to be addressed. Archival analysis of data in your IT systems may provide a sufficient assessment. Direct observation of people and processes might reveal sufficient information for action taking without further investigation.

Surveys are best used when integrated with other assessment approaches. Interviews and focus groups of key stakeholders are often needed to define the scope of a survey. Archival data analysis and direct observation, if conducted before or during the survey design phase, can provide complementary information to help refine the survey scope. Alternatively, a survey analysis might identify issues requiring additional investigation. Interviews and focus groups can probe complex issues in ways that surveys cannot easily measure. Archival data analysis and direct observation can provide data that validate the initial conclusions of a survey analysis.

Many organizations, especially large ones, conduct enterprise-wide annual or biannual employee surveys. Conducting a survey across employees in different roles doing different things involves tradeoffs. You survey diverse people from different backgrounds who experience different things at work and whose prospects for rewards, development and promotions, influence and authority, and so on are different. You have to decide to focus primarily on individual employee issues (such as motivation, turnover, etc.), business process issues (such as group dynamics, collaboration, cross-functional collaboration, etc.), or both.

For example, administrative assistants, researchers/ engineers, salespeople, laborers, truck drivers, and software programmers all have different competencies, roles, and responsibilities. They have unique career paths both internally (within your organization) and externally. Organizational processes—R&D, sales, marketing, logistics/ distribution, supply chain, IT, HR, finance, and so on— focus on very different things. If you try to use one set of questions for all employees or organizational processes, you will need to reduce the focus to the most common denominators or run the risk that entire portions of the survey will be irrelevant to the people answering it or to the leaders who have to act on it. The alternative is a survey so long it is a burden to fill out. A better solution is different surveys with different focal points for different departments.

When designing the survey sample, it is important to acknowledge the potential risks of leaving people out. If a group or unit is excluded from a survey for no logical reason and if no reasonable justification is communicated, then people might question the survey purpose and undermine its support. To mitigate this, any survey sample limits should be clearly linked to the survey strategy and communicated to the organization.

The key lies in striking the right balance. An enterprise-wide survey that tries to be all things to all people with the same questions every year is going to have significant gaps. Targeted one-time surveys of specific units, processes, or roles will always get deeper insights into the most critical current issues for those groups. The big, broad approach’s greatest benefit comes from focusing everyone in the organization on one topic in a cost-effective manner while not overselling the benefits.

Desired outcomes for the survey. A survey should never be conducted without a goal in mind. Measurement alone is not enough to justify a survey. A survey is just one step in a greater process of some kind of organizational initiative or sensing effort, such as improving morale, setting the stage for a reorganization, improving operational effectiveness, and so on.

There are many potential desired outcomes for an employee survey: improved employee retention or engagement, customer service, quality, work processes, organizational climate, change effectiveness, talent management, and more. The first challenge is selecting the highest priority outcome or set of outcomes. Effectively addressing multiple outcomes in the same survey is possible; however, if they are closely related the survey will be shorter, and both clarity of purpose and ease of responding will be greater.

For example, change may be a high priority. If the organization is about to undergo substantial change or if the goal is to assess organizational agility, then change readiness is an appropriate focus. If the organization is undergoing or recently underwent significant change, then measuring change impact likely is more appropriate. While both change readiness and change impact are aspects of change, rigorous measurement of each requires a significant number of different survey questions. Measuring both well could easily mean a long survey with little room for anything else.

For a second example, understanding employee retention is always useful. Yet retention is not equally important for all settings and roles. For roles with difficult to replace capabilities, the cost of attrition and importance of retention are high, even if turnover might be relatively low. An example of a role like this is general managers with deep organizational and cross-functional knowledge. For roles with capabilities that are easy to replace, where new entrants can quickly get to full productivity, the importance of retention is low, even if turnover is high. A role like this is a call center job for “cold call” marketing of credit cards, where a minimal amount of training is needed, turnover does not affect the productivity of other employees, and employees can get up to full productivity in a relatively short amount of time. In contrast, there are other call center jobs that are highly complex, requiring a wide and deep knowledge base and significant training and experience. In these instances, the role is hard and expensive to replace and desired retention is high.

Thus retention’s importance depends on the role’s capabilities and turnover’s impact on those capabilities and organizational effectiveness. Retention likelihood can be measured using a small number of questions on intention to turnover. Yet measuring intention to turnover is different from understanding what drives people to leave; that requires a full model including factors such as opportunities for development and promotion, pay satisfaction, supervisor support, how supportive and productive coworkers are, and more. So if retention is an important organizational priority, an entire survey easily could be dedicated to measuring the factors behind it.

The more committed leadership is to achieving the survey goal, the better you will be able to focus attention and resources on doing the measurement. However, the survey must be an impartial measurement of the situation and factors impacting the survey goal. A survey should never be crafted to lead to a predetermined outcome.

For example, suppose senior leaders want to increase the productivity of a workforce that is already working long hours and complaining informally about too much work. Someone might suggest a survey highlighting only the positive aspects of working there to show a dedicated and committed workforce ready to take on any challenge, including more work. Such a survey might focus only on readiness to take on new challenges and the opportunities for learning, development, and career advancement. Those measurements are important but tell only one-half of the story. Additional measurements of work-life balance/burnout, intention to turnover, and organizational commitment should also be included for impartial measurement that truly gauges whether people are at the breaking point and cannot handle a greater workload; with these in hand a more accurate assessment could be made of the potential negative impacts of an increased workload.

Designing a survey that impartially measures the survey goal is important for keeping employees engaged in the process and increasing participation. Employees always have some sense of the issues being addressed in a survey: it is impossible to keep the true objective hidden. If a survey is poorly designed to measure a predetermined outcome, the first employees to take the survey will realize this and spread the word among their peers. That will lead to lower response rates and increased mistrust in management for fielding the survey—the exact opposite results you want to accomplish.

Organizational levels to target. Generally speaking there are three different organizational levels: (i) individual employees or roles; (ii) teams, work groups, or functions; and (iii) business units or the entire organization. There are two separate but related organizational level issues for employee surveys: question wording and the level of analysis. Chapter three has a detailed discussion of both issues. Here we address level of analysis specifically related to survey purpose.

The types of questions that can be asked effectively vary across the levels:

✓ Issues of retention and motivation often are best addressed at the individual employee or role level.

✓ Work processes and work group climate often are best addressed at the team or work group level.

✓ Organizational climate often is best addressed at the business unit or entire organization level.

Other question types can apply across levels. For example, change readiness, change effectiveness, and perceived organizational effectiveness can be measured at each level.

Even if an issue can be addressed across levels, its importance across the levels depends on the survey objectives. Though surveys are filled out by individuals, many key insights occur at the function, work group, business unit, and enterprise levels. It is important to clarify the desired level of the organizational outcomes and adjust the survey focus accordingly. For example:

✓ Change readiness can be measured at the individual level. However, organizational change effectiveness occurs at the work group level and higher.

✓ Relationship with supervisor can effectively predict employee engagement and retention at the individual level. At higher levels, it can gauge managerial training and effectiveness.

The ultimate issue is survey length and accuracy. It is important to conduct measurements at the appropriate level that are as accurate as possible. If multiple levels measurement is a high priority, then that should be the survey purpose. If not, then use single-level measurement to minimize survey length.

Summary of Key Points from This Chapter

Recognize the limitations of surveys. Don’t overuse them. Combine them with other assessment types as appropriate (interviews, focus groups, archival data analysis, direct observation, etc.).

Choose desired survey outcomes to maximize support of top organizational priorities. Choose one or two top priorities to focus on.

Clarify the highest priority organizational level for the survey priorities. Addressing multiple levels in the same survey is doable; choosing one primary level is more manageable.