Five Lessons Learned On Using AI In HR And Compliance

Authored by Nathan Christensen, CEO of Mineral.

“Have we been replaced by AI?” That’s the question I asked our executive team this past December.

My company delivers human resources and compliance solutions to more than 1 million small and mid-sized businesses. As we watched ChatGPT become the fastest-growing app of all time, we had to confront a question we’d never faced before: Could artificial intelligence (AI) give our clients better advice than we do?

We were impressed by the technology and also confident that human expertise still had a critical role to play. To figure out exactly what that role would be, we put a popular generative AI service up against our team of HR experts. We asked it to answer a series of real HR questions asked by our customers, and then we scored the results. Here’s what we learned:

1. HR and compliance demand A+ work.

The answers provided by generative AI suffered from incorrect and incomplete information and a lack of practical applicability. Overall, we determined the AI performed like a C student. C-quality work has value, but it also can come with a cost.

Let’s take a simple example: Suppose a business has an employee who is underperforming, and the business is trying to decide how to handle the situation, including whether to terminate the employee. If the business makes the right call, no additional problems arise. If the business mishandles the situation, however, it will incur a cost of $25,000 in legal or settlement costs.

Now, let’s say the business asks generative AI for guidance, and AI gets the guidance right 70% of the time. The AI-generated guidance itself was free or nearly so. But the fact that AI gets the guidance wrong 30% of the time means there’s a hidden cost to using it: $7,500 (the error rate of 30% times the error cost of $25,000). The question for this business is whether another source would offer better advice at a price that’s less than the error costs it would save.

Reality isn’t this simple. Businesses need to balance a broad range of considerations: What are the reputational costs of making bad decisions? What is the emotional toll or distraction involved? What is the impact of the decisions on those involved? Ultimately, businesses need to look holistically at the costs and benefits of using generative AI to make decisions.

2. Asking the wrong question is risky.

Management consultant Peter Drucker is known to have said, “The most serious mistakes are not being made as a result of wrong answers. The true dangerous thing is asking the wrong question.” Drucker made this observation long before AI, but it’s more relevant now than ever.

The value of generative AI’s answers depends on the quality of the questions. The problem is that HR and compliance topics are often nuanced. Expertise is required to distill the parameters and frame the underlying question correctly.

Let’s go back to the hypothetical business above: The business might ask the question, “What are the risks of terminating an employee on three days’ notice for poor performance?” The key to answering that question is identifying all the relevant inputs, such as, “How has the business handled performance issues previously?” and, “Is the employee in a protected class?”

Without identifying those parameters and incorporating that information into the question, the business risks Drucker’s “true dangerous thing”—asking the wrong question—and receiving an answer that’s incorrect or incomplete. The power of generative AI to produce answers that appear trustworthy (even when they aren’t) makes getting the question right increasingly important, a value human experts are still uniquely capable of delivering.

3. Emotional intelligence will be a competitive advantage.

One of the recurring themes in our experience advising clients is that human experts often discover underlying or tangential issues that weren’t part of the initial question. For instance, a new policy might raise privacy or cultural concerns.

How would experts spot and diagnose issues the clients themselves weren’t even aware of? We use our emotional intelligence to observe subtle signals in communication. The power of human emotional intelligence is also relevant to delivering information. No matter how accurate or comprehensive the information is, the value it provides depends on the receiver’s willingness to use it. If we’re skeptical, fearful or resistant to information, we’re unlikely to leverage it.

An expert with keen emotional intelligence will identify hidden issues and deliver information so the person on the other end will receive, embrace and leverage it. In our tests of AI, we didn’t see evidence that it can get close to that level of sophistication—at least not yet.

4. AI won’t displace experts, but it will displace those not using AI.

As discussed above, humans have unique abilities that AI will struggle to replicate, but the reverse is also true. AI has unique abilities that humans will struggle to replicate, including the ability to synthesize vast amounts of information. Even if not trained on the full spectrum of publicly available data, AI models will have virtually infinite memory and processing power relative to humans. In our experiment with generative AI, we observed meaningful gaps in what it could do—and meaningful strengths and potential.

I believe businesses that responsibly leverage AI and enhance it with human expertise will deliver more value more efficiently than businesses that don’t. The race, therefore, is not against AI, but with AI against competitors that are not leveraging AI models as effectively.

5. Collaborative intelligence is the future.

The lessons above show that both AI and human expertise have a unique value. The true potential for businesses to elevate their HR and compliance programs will be achieved through collaborative intelligence: the intersection of human and AI. For example, leverage human expertise to define the relevant parameters, AI to gather the potentially relevant information and human expertise to refine and apply it.

It’s an exciting and fascinating time for businesses. Our experiment with generative AI exposed meaningful risks, costs, value and potential. The technology will evolve rapidly, and by identifying the unique strengths of both humans and AI and designing systems that bring them together, organizations can bring more value at faster speeds and lower costs to the businesses they serve.