AI and automated decision making

What is it?

Artificial intelligence (“AI”) is increasingly used by financial systems and financial technology (“fintech”) organizations to assist in data analysis, metrics, and decision making. Most AI systems are built and trained using statistical data and models which will result in outcomes that favour the statistical majority. In these systems, people experiencing disabilities, Deaf people, and other minority groups do not fit neatly into the model as they appear as small or insignificant data, or as statistical anomalies or outliers. People will experience this bias in different ways. This page documents some of these impacts and some possible approaches to overcome these issues.

Articles

This Articles section includes research papers, news articles, and other information published on the Internet.

• “The promise and challenge of the age of artificial intelligence” by James Manyika / McKinsey Global Institute. Describes the opportunities and challenges presented by adoption of AI including bias and privacy.

• “Artificial Intelligence and Accessibility: Examples of a Technology that Serves People with Disabilities” by Carole Martinez / Inclusive City Maker.com. Describes the possible ways AI can help people with disabilities by using facial recognition, speech adaptation, way finding, and other approaches.

• “How Artificial Intelligence Is Promoting Financial Inclusion? A Study On Barriers Of Financial Inclusion” by Anam Fazal, et. al / University of Sharjah.

• “5 obstacles standing in the way of AI adoption in financial services” by DataCamp. This article describes the five barriers to the financial service sector being able to adopt AI further.

• “How artificial intelligence affects financial consumers” by Makada Henry-Nickie / Brookings. An article outlining different opportunities in using AI for protecting and benefitting consumers, and also regulating and providing oversight for consumer finances.

• “Strategies to ensure young persons with disabilities are included in the future of work” by Institute for Work & Health. Includes a section on the impact of using AI in recruiting people with disabilities.

• “Artificial Intelligence Opens Up The World Of Financial Services” by Joe McKendrick / Forbes. Article on how AI can help improve financial services, including fraud prevention.

Experiences

This Experiences section includes stories, perspectives, and experiences submitted by people experiencing disabilities or is Deaf. Where necessary, personal information have been changed to protect the privacy of the people involved.

• Ana has been audited multiple times because her tax filing is flagged due to an unusual profile caused by her episodic disability, the audits always find her in full compliance.

Other issues being raised or addressed

• Given the degree of public attention to AI and the role of AI in financial decisions, automated and AI-guided tools have been the focus of numerous consultations. 

• Consideration of the disability barriers was distressingly absent from discussions of AI and Fintech deployment, even when the topic was AI fairness or equity. 

• test and review AI supported mortgage and loan approval processes. Discussions have begun with CRA regarding disproportionate false positive flagging for tax audits. The project team has consulted a number of AI ethics groups to discuss possible approaches to financial discrimination caused by machine learning.

External groups

The External Groups section contains links to organizations or individuals who are working, advocating, or innovating in this space.

• ForHumanity

• AI Ethicist

• Montreal AI Ethics

• Centre for Data Science at NYU

• Deep Mind group at Google

• Data Fluencies Project at Simon Fraser University

Contribute to this topic

Do you have an article or resource to add to this page? Do you have an experience you would like to share? Add your insight and perspective to this topic by using one of the following options:

• Use the comment feature on this page by either: 1) highlighting text on the page and select “Comment”; or 2) using the “Comment” section at the bottom of the page.

• Completing the article submission web form or by completing the article submission Word document and emailing it to jhung@ocadu.ca.

For your privacy and security, please avoid sharing personally identifiable information such as names and specific locations.