We posted an article recently titled “Use of Artificial Intelligence for Assessments,” which you can reference here. It discusses the guidelines and ethical considerations for the use of assessments for selection or development. The current article expands on this and highlights the legal considerations in using AI in HR assessments. This article is not meant to convey legal advice, but rather to summarize information that HR professionals should consider when using assessments that integrate AI. They should also always ensure that they are consulting with their company’s own legal function about the assessments and assessment-related processes they are using for decision-making purposes.
Primary Legal Considerations
The primary legal considerations for AI use in HR assessments include:
Privacy or Data Security
Privacy and data security are at the top of our HR customers’ minds when they are implementing assessments of any kind – for selection or development purposes. They want to understand how assessments which integrate AI protect the personal data of the employees in their organizations.
Privacy and data security have been a concern for a very long time, even before the use of AI in assessments. Firms providing assessments need to ensure that they are abiding by data security regulations and guidelines, such as the EU Data Protection Laws (EU General Data Protection Regulation - GDPR 2016/679), and when applicable, the data protection or privacy laws of any other country.
When AI is used in an assessment, there are additional assurances that a company should have from their assessment providers. For example, the provider should not be using AI technologies that use the company’s data to improve their models. Neither the data in prompts nor the output from the Large Language Model (LLM) used should be stored or leave the environment (LLMs are AI models that have been trained on a massive number of text documents in order to achieve general-purpose language generation capabilities).The inputs to the LLM should only be used to generate outputs that would be returned to the assessment provider’s application.
It is important for companies to have their legal function review the specifics of an assessment provider’s agreement to ensure these protections are in place.
Bias or Discrimination
As mentioned in our previous article (accessible here), any assessment used for decision-making purposes should follow the Uniform Guidelines on Employee Selection Procedures and the Principles for the Validation and Use of Personnel Selection Procedures, published by the Society for Industrial and Organizational Psychology (SIOP). SIOP also recently published Considerations and Recommendations for the Validation and Use of AI-Based Assessments for Employee Selection.
Assessments which integrate AI can incorporate biases in various ways, including discriminatory training data and algorithms used in building the AI model, as well as cognitive biases that could be inherent with the developers. SIOP’s recent Considerations and Recommendations article outlines the steps that should be taken to ensure the reduction and absence of bias and discrimination in assessments which integrate AI.
Errors or Misstatements
Most users of AI in general are aware that there is a risk of inaccuracies or fabrication with AI. Assessments that use AI to elaborate on assessment conclusions or offer advice to users based on questions asked, should:
Pertinent AI Regulations
Governments are beginning to introduce their own legislation to address AI-related risks.
For example, in the United States, there is no comprehensive federal AI legislation yet, but there are state and local laws in place. For example, New York City requires bias audits and transparency for Automated Employment Decision Tools (AEDTs). Colorado SB 24-205 regulates high-risk AI systems and mandates bias audits. In Illinois, before an employer conducts an AI interview or, more generally, uses AI to make employment decisions, the employer must notify the applicant that AI will be used. Employers that rely on AI to analyze video interviews must also obtain prior consent from the candidate to use the AI program and report demographic data about the applicants to the Illinois Department of Commerce and Economic Opportunity.
In the European Union, the EU AI Act was finalized in 2024 with phased implementation through 2027. The EU AI Act classifies AI used in employment related decision-making as high-risk, requiring things like bias mitigation and human oversight and transparency.
Companies and HR departments using assessments for decision-making purposes that integrate AI should be familiar with the regulations pertinent to them. Assessment providers who are often providing assessments on a global basis need to understand the legislation in place anywhere that they are doing business.
In addition, many companies have their own guidelines in place for AI governance. HR professionals should be familiar with these guidelines and how they impact their decisions related to assessments as well as overall processes that involve decision-making (e.g., selection, performance management, promotion).
What About the Benefits?
At first, the legal considerations may seem overwhelming. However, HR professionals and companies who use assessments have needed to adhere to guidelines and regulations concerning the use of assessments for decision-making purposes for decades. They have also had to ensure data security and privacy. These considerations are nothing new, and they will continue with the use of assessments for decision-making purposes that integrate AI. In addition, with the use of AI, factors like ensuring transparency regarding AI use, and abiding by any pertinent AI-related regulations come into play.
Governance regarding any use of assessments, including assessments integrating AI, is critical for organizations, and should be common practice. Ensuring this governance allows for many benefits associated with AI use in HR assessments, including:
With these benefits and more, companies, and each function represented within, including HR, are deep into exploring all of the ways AI can benefit them and their stakeholders. AI is here to stay and used with the right governance has much to offer. This, of course, has always been the main thought about AI – that human oversight and governance is required to ensure the results are what we envision and need.
Conclusion
This article focused on legal considerations for the use of AI in HR-related assessments. It also briefly conveyed the benefits. The next article will focus more on the benefits and provide an example of the use of AI in the evolution of an assessment.
Follow us on LinkedIn to get immediate access to the next article and all of our other resources.