All the discussion in the popular press about artificial intelligence (AI), machine learning, cryptocurrencies, and the gig economy can be both overwhelming and more than a bit intimidating. Some of that sense of intimidation comes from the fact that these terms are often bandied about by folks who do not understand (or chose to obfuscate) the underlying technology, and some of it purely from the underlying technology.
It is indisputable that these technologies affect so many aspects of society and culture that it will soon be impossible to practice as an attorney without some—if not many—of these technologies directly affecting our practices. In this article I will take just one of these topics, AI, and discuss some of the underlying technology and the legal and licensing challenges it can pose.
The popular press is very fond of using the terms “AI” or “intelligent systems” without distinguishing among the different areas of research that fall within the broad description of AI. The advent of ever-faster processors, almost free memory, and sophisticated search algorithms has allowed massive advances in some areas of AI—for example, pattern recognition, machine learning, and neural nets—but other areas such as natural-language processing still pose incredible challenges. For simplicity’s sake, let’s hone in on machine learning, and in particular on neural networks.
Neural networks are programs that can be trained to be extremely good at pattern recognition. Without getting into the nitty-gritty of neural networks, the basic principles are as follows:
- You provide the neural net with a learning set: a large collection of items related to the pattern you want the neural network to recognize for which you tell the program the correct answer.
- The program learns patterns—primarily by setting weights for parameters—from the learning set and then you run the program on a test set: another large collection of items for which you know the answer. You provide the program its results and the correct answers.
- The program recalibrates parameters; i.e., re-learns the pattern.
- You rinse and repeat until the neural network starts providing answers within an acceptable error range.
Neural Network Liability
A couple of problems which hinder neural networks, and that one can identify without thinking too hard, are they are only as good as: (a) the training and test sets, and (b) the definition of an “acceptable error range.” For example, if you (as the programmer) decide you are more concerned with false negatives than with false positives, you will get a neural network (or to use the popular vernacular an “AI”) with very different characteristics than if you are more concerned with false positives.
It is not surprising that companies and entities developing neural networks for security and law and order purposes are more concerned with false negatives than false positives. In July 2018, the ACLU conducted a test of Amazon’s facial recognition software called “Rekognition.” Rekognition incorrectly identified 28 members of Congress as having been arrested for committing a crime. In other words, Rekognition showed a bias toward avoiding false negatives in favor of providing false positives. Further acerbating the situation, Rekognition disproportionately identified members of Congress of color as having previously been arrested for having committed a crime, including six members of the Congressional Black Caucus.
The problems arising from “faulty” training sets and test sets are best illustrated with Amazon’s own troubles with its résumé/candidate-screening software. As Reuters reported last year, Amazon’s machine learning group had been working on a neural network to screen résumés “with the aim of mechanizing the search for top talent.” The problem was that most of the good matches used in the training set were men. The neural network thus “learned” that it should screen out résumés that referred to the candidate having been the captain of the “women’s chess team” or having graduated from a women’s college. An AI could very easily display similar biases against candidates with non-Anglo-Saxon names.
Amazon got a bit of a black eye from its failed experiment with its résumé screening software. A smaller startup that decides to rely on a platform created and maintained by a third-party vendor may not be so lucky. In fact, it may face civil and criminal penalties, or even discrimination lawsuits from rejected candidates. To avoid these potential risks, it is critical that companies entering into agreements with such vendors ensure that the agreement properly allocates liability for potential biases in the AI. Of course, the agreement should also include appropriate obligations to defend and indemnity provisions.
It is unquestionably the case that we interact with systems which rely heavily on pattern recognition and machine learning every day (not to mention the other technologies mentioned at the beginning of this article). The convenience and efficiency that many of these technologies bring to our daily lives can, however, blind us to the legal issues and concerns they raise.
