The Need for a Hermeneutic of Suspicion (A.K.A. Why Wes Doesn't Fully Trust Qualitative Research Analysis)

(You can watch this presentation by clicking the below link or read about it here)

It's Not Entirely My Fault

Growing up with a professional communicator and statistician for a mother changes much about one's childhood. For example, I learned very early on that if I had to convince my parents of something they would be naturally disinclined toward, I needed to prepare my arguments in advance and be ready to defend my stances cooly and confidently. Nine times out of ten, I was rewarded more for proper speech skills in building a compelling case than because I had actually managed to sway her opinion. 

And, in critical, win-or-die situations, I learned it was always best to convince Dad instead. 

One particular lesson that has proved invaluable throughout my life is a technique that my mom would regularly use against me: words, numbers, and data can too often be manipulated to say almost anything you want them to say. Every time I would bring a case to my parents about why I needed this new toy--or, later, electronic device--or why we needed to take a trip to a particular place, I would have to be very careful not to use data that could be flipped on its head. 

"Jimmy's parents just got him a Nintendo64! That makes eight of my friends who have one, and I'm struggling to keep my NES from overheating."

"Yes, but Jimmy's parents both work for the bank and make more money than we do. And besides, of the twenty-four kids who came to your birthday party, only a third of them have these new systems. Half don't have a Nintendo at all, so it seems like you are still better off than others."

(When I finally did convince her, it was only after I had saved up the money to pay for it myself. Which, in retrospect, might have been her plan along...)

The Holy Grail of Gaming
Céréales Killer, CC BY-SA 3.0 <http://creativecommons.org/licenses/by-sa/3.0/>, via Wikimedia Commons

Statistical data, especially as Kid-Wes used to use it, can easily be flipped on its head. 

Now, this does not mean that there is no such thing as quality quantitative research. The academic and professional worlds are replete with example after example of researchers doing the world to ensure that their data is as complete and variable-free as possible. Using the scientific method and leaning on each other for peer review, these people produce compelling results that must be taken seriously and are used to enhance our understanding of the world around us. Instead, my mom was teaching me to always engage such research and data with what Paul Ricoeur calls a hermeneutic of suspicion: interpreting everything with the understanding that it may not actually be saying what we think it says or what we're told it says. 

Engaging Quantitative Research with a Hermeneutic of Suspicion

In a post-pandemic world where we experienced what it is like to be bombarded with the extremes of "trust scientists" and "do your own research", developing a healthy hermeneutic of suspicion can be our saving grace. This is more than simply taking a researcher's word as gospel truth. Still, it is not going so far as to think that the average layperson can replicate and enhance the work that went into drawing educated and peer-reviewed conclusions. 

Public Domain

A proper hermeneutic of suspicion asks questions like: 

• "What else could the data be saying?"  
• "What other ways could the survey questions be interpreted that might change the resulting data?"
• "What other conclusions might have been drawn from the same research?"
• "What variables might not have been taken into account?"
• "What other data might be helpful in this conversation?"

This approach to research and data does not guarantee that you will always come to correct conclusions, and it does not mean that you will always end up disagreeing with the original research team. Instead, a healthy hermeneutic of suspicion allows individuals and groups to lean into curiosity while recognizing and respecting the work that others have already poured into the process. As the great coach and manager of AFC Richmond so famously said, "Be curious, not judgmental." Engaging the world with a hermeneutic of suspicion does just that. 

5 Examples for the People

The concept of the hermeneutic of suspicion is not abstract; it is extremely practical for individuals at any and all levels of education and expertise. In the below articles, it is easy to see how this interpretation tool (1) helped guide the researchers' work and (2) help the reader draw robust conclusions and guide further reading. 

The Effects of Authentic Leadership on Athlete Outcomes: An Experimental Study

Summary: Malloy, Kavussanu, and Mackman seek "to examine the effects of authentic leadership on athletes' trust, enjoyment, commitment, and a range of morally relevant variables" (p. 31). They surveyed 129 sports science student-athletes from a specific British University. Their method used three scripts that each described a coach with different characteristics: one with high authentic leadership scores, one with low scores, and one to serve as a mid-range or control script. Student-athletes were then asked scaled questions to determine how they felt and/or would react to each coach.

Hypotheses:

1. Comparatively, "participants in the high authentic leadership condition would: anticipate feeling greater trust and enjoyment; be more committed to play for the described coach; be less likely to cheat and be aggressive; and anticipate feeling more guilt for cheating and aggression" (p. 31).
2. The opposite effect would be true for the low authentic leadership condition.

Results:

Our researchers concluded from their data that Hypothesis 1 was half-supported, showing that the authentic leader condition expected greater levels of trust, enjoyment, and commitment but did not show a significant enough effect on cheating, aggression, or guilt toward either. Hypothesis 2 was also supported with the above caveat. 

Hermeneutic of Suspicion: While the above data clearly shows a distinction between the High AL and Low AL, the researchers do not name or discuss the similarities between High AL and the Neutral AL (control) condition. The data would seem to show that Low AL produces poorer results in the given categories, while High AL does not necessarily produce significant positive results. The researchers also include their own hermeneutic of suspicion by including a section on "Study Limitations and Future Research Directions" (p. 38). Combined, these show that there is more work to be done and that this research is not the final word on authentic leadership in coaching. 

Rebellion Under Exploitation: How and When Exploitative Leadership Evokes Employee's Workplace Deviance 

Summary: Lyu, Wu, Ye, Kwan, and Chen seek to show that "under leader exploitation, employees engage in organizational and interpersonal deviance", study how this has a mediating influence on moral justification, and define who is impacted the most from this type of toxic system (p. 484). To do this, they surveyed workers from two different service companies in China: a logistics company and a hotel. 329 of the original 550 surveyed completed the three questionnaires, each spaced two months apart. Those surveyed answered questions on a sliding scale to from 1 (strongly disagree) to 5 (strongly agree). 

Hypotheses:

1. "(a) Exploitative leadership is positively related to employees' organizational deviance. (b) Exploitative leadership is positively related to employees' interpersonal deviance" (p. 486).
2. "(a) Moral justification mediates the association between exploitative leadership and organizational deviance. (b) Moral justification mediates the association between exploitative leadership and interpersonal deviance" (p. 487).
3. "Hostile attribution bias moderates the association between exploitative leadership and moral justification such that the association is stronger for employees with high (versus low) levels of hostile attribution bias" (p. 488).
4. "(a) Hostile attribution bias moderates the indirect effect of exploitative leadership on organizational deviance through moral justification, such that the indirect effect is stronger for employees with high (versus low) levels of hostile attribution bias. (b) Hostile attribution bias moderates the indirect effect of exploitative leadership on interpersonal deviance through moral justification, such that the indirect effect is stronger for employees with high (versus low) levels of hostile attribution bias" (p. 488).

 Results: Our researchers found a significant correlation to exploitative leadership and employee deviance...

...a significant correlation to moral justification and employee deviance...

...exploitative leadership and hostile attribution bias was significantly linked to moral justification...

 

... and the conditional influence of exploitative leadership on employee deviance via moral justification was significant among those with high hostile attribution bias...

...meaning that all hypotheses were supported by qualitative data research. 

Hermeneutic of Suspicion: While the above data shows clear linkages between exploitative leadership and the deviance/corruption of employees, the scope of this project has specific drawbacks. Our researchers did a wonderful job in naming the three largest of these in their "Limitations and Future Research" section (p. 496). This study is best read in conversation with other pieces of work that (1) utilize a longitudinal study to decrease bias in those surveyed, (2) name and address other toxic leadership categories such as pseudo-transformational and despotic leadership as well as strategic bullying, and (3) engage a global population to remove some of the cultural biases specific to China. This study is meant to continue a conversation, not provide the final statement of it. 

Role of Organizational Justice in Linking Leadership Styles and Academics' Performance in Higher Education 

Summary: Khan, Gan, Khan, and Saif seek to examine the association between transformational/transactional leadership styles, organizational justice as mediator, and academics' performance. They use inferential methodology to draw correlative conclusions based on their sample survey, in which employees rated their bosses by answering different questions on a sliding scale of 1 (strongly disagree) to 5 (strongly agree). Their sample size was 285 academicians from two Pakistani universities. 

Hypotheses:

1. Significant associations exist between transformational/transactional leadership styles, organizational justice as a mediator, and academics' performance.
2. Organizational justice serves as the mediator between transformational leadership and academics' performance.
3. Organizational justice serves as the mediator between transactional leadership and academics' performance.

Results: Our researchers found a higher correlation between transformational leadership and academics' performance than with transactional leadership and found that organizational justice served at least as a partial mediator to performance for both transformational and transactional leadership styles. Since a correlation was determined with both leadership styles, all three hypotheses were at least partially supported. More importantly, for continued research, the results show a clear trend toward transformational leadership. 

Hermeneutic of Suspicion: While our researchers' work here seems fairly straightforward, and their data drives the results well, their conclusions are a bit of a stretch. After surveying 285 Pakistani academics--both professors and department heads--they reach the conclusion that transformational leadership is both desired by and crucial for academic leaders throughout developing countries. They go so far as to state that "higher institutions are required to implement the transformational style of leadership in letter and spirit to attain the desired outcomes from academics" (p. 14, emphasis mine). Their lack of a "limitations of research" section also leaves something to be desired for the reader.

Serving While Being Energized (Strained)? A Dual Path Model Linking Servant Leadership to Leader Psychological Strain and Job Performance

Summary: Li, Chen, Bai, Liden, Wong, and Qiao work to develop a dual-path model that helps identify the psychological benefits and strains of the servant leadership model and identify the proper boundary conditions that lead to healthy servant leadership. They used the longitudinal method, sending out five assessments in one-month intervals to team leads, team followers, and team bosses of a large Chinese manufacturing company. They received all five assessments from 474 leads, 3712 followers, and 97 bosses. 

Hypotheses:

1. "(a) Servant leadership is positively related to leader need satisfaction; leader need satisfaction mediates (b) servant leadership's negative relationship to leader psychological strain and (c) its positive relationship to leader job performance" (p. 661)
2. "(a) Servant leadership is positively related to leader role conflict; leader role conflict mediates (b) servant leadership's positive relationship to leader psychological strain and (c) its negative relationship to leader job performance" (p. 662).
3. "(a) Leader-leader exchange (LLX) strengthens the positive relationship between servant leadership and leader need satisfaction. (b) LLX weakens the positive relationship between servant leadership and leader role conflict" (p. 662).
4. "(a) LLX strengthens the negative indirect relationship between servant leadership and leader psychological strain through leader need satisfaction. (b) LLX strengthens the positive indirect relationship between servant leadership and leader job performance through leader need satisfaction" (p. 663). 
5. "(a) LLX weakens the positive indirect relationship between servant leadership and leader psychological strain through leader role conflict. (b) LLX weakens the negative indirect relationship between servant leadership and leader job performance through leader role conflict" (p. 663). 

Results: Our researchers found that servant leadership was positively related to leader need satisfaction and leader need was negatively related to leader psychological strain but not to leader job performance. This supports hypotheses 1a and 1b but not 1c. 

Interestingly, this is the only study of the five here where an entire multi-pronged hypothesis point was not supported by the data. Counter to Hypotheses 2a-c, the data showed no relation between servant leadership and leader role conflict, and the indirect effects of servant leadership on leader psychological strain and leader job performance was insignificant. 

The LLX strengthened the positive relationship between servant leadership and leader need satisfaction, and the relationship between servant leadership and leader role conflict was positive when LLX was low but turned negative when LLX was high.

The LLX  strengthened the negative indirect relationship between servant leadership and leader psychological strain through leader need satisfaction. There was no significant evidence to support the LLX strengthening the positive indirect relationship, though, so whereas Hypothesis 4a was supported, 4b was not. The data did show significant indirect relationships between servant leadership and leader psychological strain (5a) and leader job performance (5b) as moderated by the LLX. 

Hermeneutic of Suspicion: This research is extensive and targeted in the questions that it seeks to answer. The unsupported parts of the multi-point hypothesis show that there is more research and work to be done on the topic of servant leadership, its psychological strain, and leader job performance. The longitudinal method has given the researchers more than enough data to continue to comb through, so this work could still result in more unearthed information. The authors' "Limitations and Future Research Directions" also shows that they are well aware of the gaps in their research and establish a trailhead into the next research endeavor. 

Linking Servant Leadership to Follower Emotional Exhaustion Through Impression Management

Summary: Peng, Gao, and Wang seek to learn about follower well-being as a primary outcome of leadership behaviors and communicate how this research can be used by organizations to benefit the training and development of servant leaders. They did this by conducting two three-wave studies. The first, which focused solely on Hypotheses 1 and 2, was done at a large regional bank in China, where 197 workers completed three surveys six weeks apart. Unhappy with the results and worried that the data of this first study were skewed based on the homogeneity of the participants, the researchers replicated the study in the United States with a larger and more diverse sample population and only one week between surveys. This second study covered all four of the hypotheses below, surveying 377 full-time employees in the USA. 

Hypotheses:

1. "Servant leadership will be positively related to (a) exemplification, (b) ingratiation, and (c) self-promotion" (p. 646).
2. "Servant leadership will have a positive and indirect relationship with employee emotional exhaustion as mediated by (a) exemplification, (b) ingratiation, and (c) self-promotion" (p. 646).
3. "The positive associations of servant leadership with (a) exemplification, (b) ingratiation, and (c) self-promotion will be accentuated by POP" (perceptions of organizational politics) (p. 647).
4. "The indirect and positive relationship between servant leadership and emotional exhaustion will be strengthened by POP. Specifically, the indirect effects of servant leadership through (a) exemplification, (b) ingratiation, and (c) self-promotion will be stronger among employees who perceive higher POP" (p. 647).  

Results: Our researchers discovered that servant leadership was positively related to exemplification, ingratiation, and self-promotion, all of which support Hypothesis 1...

... Servant leadership's indirect relationship with emotional exhaustion mediated through exemplification and ingratiation was positive, but its relationship mediated through self-promotion was not significant, supporting Hypotheses 2a and 2b but not 2c...

 ... The interaction between servant leadership and POP was significant in accentuating all three variables, supporting Hypothesis 3...

... and that the indirect and positive relationship between servant leadership and emotional exhaustion was strengthened by POP through the effects of exemplification and ingratiation but not through self-promotion, supporting Hypotheses 4a and 4b but not 4c.

 

Hermeneutical Suspicion: The researchers show great character in recognizing the limitations and issues with the first study and naming their hermeneutical suspicion as they analyzed the data. Another place that brings me pause is that the researchers decided to use the Study 2 participant numbers after the second wave of surveys, totaling 377 completed. Of those 377, only 302 completed all three surveys, leading to 75 surveys needing more essential data. The researchers decided to weight these incomplete surveys differently and still use the data, knowing this could potentially skew the results. In a desire to reach a necessary sample size, they compromised the methodology and process. They should have included this caveat in their "Limitations and Future Research" section, where they also lift up the sample size and cultural differences between studies 1 and 2. 

So... What?

Stephen Amell's Oliver Queen re-popularized the Russian saying, "Trust but verify." This is precisely what we do when we correctly engage in research and study. Quantitative data-driven research is extremely important and should be approached with this in mind. More brilliant minds than my own have given of themselves to pour over vast amounts of information in order to distill it down to palatable bites for me, and I am grateful for them. 

And.

Numbers and data can be manipulated to say almost anything we want them to say. A healthy hermeneutic of suspicion helps us keep this in mind, allowing us to better digest and utilize that information. And, if we're lucky and well enough prepared, it just may land us a snazzy new Nintendo 64...