It's about time: measuring the pulse of engagement

Citation

Winton, S. and Palmer, S.N. (2018), "It's about time: measuring the pulse of engagement", Strategic HR Review, Vol. 17 No. 3, pp. 164-165. https://doi.org/10.1108/SHR-01-2018-0010

Employee opinion surveys, commonly branded as measuring employee engagement (EE), are a key tool for HR strategic efforts. From an HR perspective, it is essential that EE survey data be useful, actionable and accurate. Unfortunately, self-reports are notoriously flawed. Schwarz (1999) contended that self-reports are often as much about the questions as the answers. For HR practitioners implementing EE surveys, this means that careful attention must be given to questionnaire design issues such as item order, question wording and response options. Given the dynamic, fluctuating nature of EE (Bledow et al., 2011; Reis et al., 2016), we contend that one element of employee opinion surveys is often overlooked – a clear and explicit understanding of time.

The intention of HR efforts to measure, track and act on EE is contingent on it being time-bound. There is a clear difference between being an engaged employee and being engaged in the moment, the past week or the past month. This distinction is all the more important considering that many organizations are beginning to supplement their annual or biennial survey with smaller, more frequent pulse surveys as organizations pursue more timely, actionable data (Van Rooy et al., 2011; Welbourne, 2016). Such surveys cannot, and should not, rely on self-reported measures that purportedly measure a stable characteristic. If the survey data are not accurately capturing the attitudes and behaviors of employees, then recommendations or action plans could be addressing the wrong issues. For HR leadership – especially as organizations move to more frequent pulse surveys – it is critical to understand how steps can be taken to improve the accuracy of measurement in current survey efforts.

Measuring the pulse of the engaged workforce

HR practitioners tasked with measuring EE have generally neglected time-bound survey items. The measurement of EE is too often left at the discretion of the respondent to interpret the length of time the measure is alluding to. For example, take the item “I am often inspired to do my best work.” Without a frame of reference, this measure may implicitly suggest that engagement is stable over time and can be captured by an individual thinking about “often.”

For organizational survey efforts, adding a time frame to a question prompt could potentially be used as a means of limiting social desirability and self-presentation bias. Consider an item such as, “I am always ready to help my coworkers.” Such items essentially ask the rater to make a global evaluation of their behavior or, in this case, helping behavior (i.e. I am an altruistic person). If, on the other hand, the reference of time is added, the behavior can now be perceived as something that varies (e.g. “Past month, I regularly helped my coworkers”). The respondent does not necessarily make a global evaluation if they rate the item low and the potential for bias weakens (i.e. inadvertently inflating one’s report of altruistic behavior). Consider how the addition of a time-based frame-of-reference alters the following items:

• I find my job enjoyable and challenging.

• During the past two months, I found my job enjoyable and challenging.

• Generally speaking, I am enthusiastic about my work.

• Comparatively speaking, this past quarter I was enthusiastic about my work.

• My job consistently makes good use of my skills and abilities.

• Past month, I was able to utilize my skills and abilities.

• I am always able to maintain a balance between work and personal life.

• During the past three months, I have been able to maintain a balance between work and personal life.

The inclusion of a time-based frame of reference might not be relevant for all survey items, but a time frame should certainly be considered for EE items related to attitudes, feelings and behaviors. Further, when it does not make sense to include a time-based reference to the item stem, HR practitioners should consider adding instructions, and potentially scales, that frame the time period. In fact, such statements should be added regardless of whether or not a time frame is utilized as a means to limit social desirability. For example:

• As you rate the following items, remember that there are no wrong answers. It is important for you to indicate how often, in the past quarter (3 months), you experienced the following at work.

• It is important to rate how you feel, not how you think you should feel. For the following items, rate how you feel about your work conditions this past month as compared to a typical month.

A final point of consideration includes determination of an appropriate frame of reference. We suggest framing according to organizationally relevant intervals, such as when pulse surveys are administered or around important organizational/team goals or cycles. Beginning with specific questions to answer regarding engagement (e.g. “Have engagement levels on my team changed since the launch of the XYZ initiative?”) can help guide both the formation of survey item timeframes and the data collection process overall (i.e. measuring engagement over multiple time points to examine changes).

Conclusion

Practically, organizations should be aware that the inclusion of a time reference altogether, as well as different frames of reference (e.g. monthly, quarterly) will change the type of data collected, which could ultimately lead to more accurate conclusions about the workforce’s engagement. HR practitioners should strongly consider how the time frame, or lack of time frame, could be impacting how individuals respond to it. If engagement is not being properly measured, it will be difficult to make accurate conclusions, which could subsequently limit the effectiveness of action planning efforts. It truly is “about time” to consider time-based references in survey items, especially pulse surveys measuring EE, to provide accurate and actionable data for HR strategic efforts.