The seminar AI in Medicine in Context: Hopes? Nightmares? was held at the Centre for Ethics at the University of Toronto on October 17th, 2017, with special guest Dr. Sunit Das. Dr. Das is a scientist at the Keenan Research Centre for Biomedical Science, a neurosurgeon at St. Michael’s Hospital, and an assistant professor in the Department of Surgery at the University of Toronto. Dr. Das discussed ethical questions and concerns regarding the impact of artificial intelligence (AI) in law and medicine. An especially interesting topic that he discussed was how AI may fail to make the most ethical decision, which may result in an undesirable outcome that negatively affects a person’s life.

In the seminar, Dr. Das discussed an interesting case from the United States, where AI was used in the legal system with harmful results. AI was used in a court case to determine the length of an accused person’s sentence, and recommended that the accused serve more time in prison. A lawyer challenged this decision and conducted his own investigation into the AI. He discovered that the AI recommended sentences that depended on the accused’s race, which was prejudicial and racist. This is a serious issue: how can the legal system recognize when AI is using racial and socioeconomic factors in sentencing decisions, rather than past criminal involvements and activities? As we have seen in this case, the AI made an unethical decision that would have kept the accused person incarcerated for additional years of his life because of his race.

Seeing as AI made the wrong decision in the legal field, it is not far-fetched to imagine it making unethical decisions in a medical context as well. Indeed, Dr. Das argued that such decisions would increase current racial and gender disparities in our health care system. If AI analyzed biased data, which exist in scientific research, then it could potentially make an unethical decision because it would not be able to distinguish wrong from right.

To prevent such issues from occurring in legal and medical fields, ethical consideration and regulation of AI is needed. AI should be developed and implemented gradually to ensure that mistakes such as analyzing biased data are avoided. Dr. Das concluded that even if AI is more accurate in empirical decisionmaking, it is not always the case that AI is the ethical and moral way forward. For this reason, it must be carefully used.