Consultation: Ontario’s Trustworthy Artificial Intelligence (AI) Framework

Overview

Building a digital economy that is powered by trustworthy artificial Intelligence (AI) is a key goal of the government’s Digital and Data Strategy. Already, researchers and entrepreneurs are at the forefront of scientific AI advancements, unlocking new ways to improve Ontarians’ lives and foster long-term economic growth. These technologies also have the potential to help the government transform vital programs and services to better serve people, while also saving them time and money.

However, as with all new technologies, the use of AI must be rooted in democratic principles and individual rights. To protect Ontarians, the province is developing a Trustworthy AI Framework to support AI use that is accountable, safe and rights based. To start, we will create guidelines for the government’s use of AI. This framework, created with your help, will outline how the government will use AI responsibly to minimize misuse and maximize benefits for Ontarians.

Ontario’s AI framework will be developed following Open Government Partnership principles to demonstrate the province’s commitment to transparency, accountability and working in the open with the people of Ontario. As an active member of the Open Government Partnership (OGP) since 2016, the public development of this AI framework and its resulting action items will form Ontario’s 2021-2022 OGP Action Plan.

Highlights of the consultation

During the consultation, we asked you:

• for input on how the government can ensure AI is used responsibly to minimize misuse and maximize benefits for Ontarians.
• to share your ideas to improve the public’s trust in AI. Your experiences, concerns, and insights will help the government make its AI use.

You told us that:

1. You are concerned about the protection of your privacy and your private data.
2. You want to know when we are using data-related technology, like algorithms, automated decision-making, or AI, and what we are using it for.
3. You want the right to contest the results of algorithmic decision-making if harm has been done and to ensure that human review is available when requested.
4. We should provide definitions for key terms and use plain language to communicate our AI use.
5. More accountability and oversight of how and when these technologies are used by government is needed.
6. You want to be involved in decisions related to how AI is developed, when it’s used, how it is trained.
7. You want government to:
   • Review and audit data-related technologies regularly while they are in use, to ensure risk doesn’t creep in over time.
   • Have clear rules, processes, and tools to ensure the safe and secure use of data-related technology to better serve and support people, communities and businesses.
   • Make decisions based on specific situations and their unique circumstances.

What we heard

Commitment 1: No AI in secret

What we heard

“AI and algorithms can be difficult to understand and difficult to explain. The “black box” issue (that the public does not see how a decision was made) can prove challenging for trust in government decision-making. Disclosure and transparency of AI systems is important for public confidence in government generally and AI systems specifically. Disclosure and transparency are also necessary to meet procedural fairness obligations, provide access to justice, and to uncover potential discriminatory outcomes, software errors, or inaccurate or incomplete data.”

• You’d like help understanding what Artificial Intelligence (AI) is and how it can affect people’s lives.
• You’d like to know when data-related technologies like algorithms or AI are being used, how they work, and what data is being used by them.
• You want human intervention in processes using AI when the decisions can have a serious impact on your lives.
• You asked to ensure people can challenge and change unfair decisions by automated systems.
• A list or inventory of the technologies and data-related tools used by the government would help people understand what AI use really looks like.
• You want the rules that apply to the government to also apply to companies when they are providing services to the government.

Survey Results

During the consultation we asked you to rank a list of potential actions we could take to ensure Artificial Intelligence technologies are developed transparently.

Survey respondents ranked the following actions, from most to least important.

1. The most important action according to respondents was: Create accountability for the use of AI in the government by giving people rights to address potential biases created by the AI (for example, the right to plain language, right to contest, and right to opt out).
2. The second most important action according to respondents was: Be fully transparent when using algorithms to interact with the public (for example, rules that require informing the public if they are interacting with a machine or have decisions made about them by an algorithm).
3. The third most important action according to respondents was: Provide clarity and transparency to the public on how Ontario collects data for use in algorithms (for example, explore options to update provincial notices of collection to inform the public if data collected is used to develop algorithms for decision-making).

Commitment 2: AI use Ontarians can trust

What we heard

“Set out in the legislation and Framework a recognition of human rights values and principles, and commitment to address systemic bias in AI that negatively affects or fails to appropriately account for the unique needs of Code-protected groups including vulnerable populations such as people with disabilities, children and older persons, Indigenous and racialized communities, as well as low-income communities”

• Any Artificial Intelligence (AI) tools used should be fully and clearly understood by the people using them.
• You want the government to carefully consider when we should and should not use AI.
• Independent review of AI tools and their use is important to earning and keeping your trust.
• In order to trust the government’s use of data-related technologies like algorithms and AI, you need to trust the data that the government uses to create it and operate it.
• Trust in government use of data-related technologies like algorithms and AI in decision making processes requires regular review and auditing of the data and algorithms informing the decisions.
• You want the government to ensure that our use of data-related technologies and tools, including AI are doing what they are supposed to be doing - and doing it well.
• You want to be able to check that we are following our own rules.

Survey Results

During the consultation we asked you to rank a list of potential actions we could take to ensure we use Artificial Intelligence (AI) in a way Ontarians can trust.

Survey respondents ranked the following actions, from most to least important.

1. The most important action according to respondents was: Ensuring processes are in place so that algorithms are continuously tested and evaluated for bias/risk and whether audits or human oversight controls are needed.
2. The second most important action according to respondents was: Deliver recommendations on ways to update Ontario’s rules, laws and guidance to strengthen the governance of AI, including whether to adopt a risk-based approach to determine when which rules apply.
3. The third most important action according to respondents was: Assess whether to use an algorithmic assessment tool as a way to measure risk, security, and quality.

Commitment 3: AI that serves all Ontarians

What we heard

“We encourage the government to view AI as a tool to better understand the needs of vulnerable people and improve their conditions. In keeping with the goal of “AI that serves all Ontarians,” AI should be made accessible to everyone by design so that the positive impacts can be more widely felt.”

• You want Artificial Intelligence (AI) systems to work for people without easy access to technology.
• You want the government to fairly assess the risk of using data-related tools, like AI, to the people they serve – especially vulnerable populations.
• You want a clear and simple way to provide input, challenge decisions and impact the choices we make about government use of data-related technologies, like algorithms and AI.
• You want the government to provide alternatives to data-related digital services that include AI without accepting a lower quality of service.
• You want government to:
  • Talk to you about how using data-related technology, like AI, will work for you before, while and after we create them
  • Use plain language and clear descriptions when talking about AI
  • Address bias and systemic discrimination in AI systems so the decisions of the future can be fair and equitable.
  • Create incentives for the private sector to develop ethical tools that the government can use.

Survey Results

During the consultation we asked you to rank a list of potential actions we could take to ensure we develop Artificial Intelligence to serve all Ontarians.

Survey respondents ranked the following actions, from most to least important.

1. The most important action according to respondents was: Embed equity and inclusion in the use of data and digital tools by requiring organizations to take steps to mitigate potential harms (for example, data set requirements, documentation requirements for traceability, accountability provisions).
2. The second most important action according to respondents was: Assess whether the government should prohibit the use of AI in certain use cases where vulnerable populations are at an extremely high risk.
3. The third most important action according to respondents was: Engage with sector leaders and civil society to develop a standard for “trustworthy AI” and a process to certify that vendors are meeting the government’s standard.

Draft commitments and actions consulted on

The following are the draft commitments and actions we consulted on from May 7 to June 4, 2021.

Commitment 1: No AI in secret

Problem

For people to trust that AI is safe and appropriate, they must first be aware of the government’s use of AI. As a result, the government needs to be transparent about how, when, and why these tools are used so that people have a right to address potential biases created by the AI algorithms.

Future success statement:

The government is transparent with the public about its use of algorithms to process applications and the public is supported by the right to contest a decision if an error is made.

Commitment 2: AI use Ontarians can trust

Problem

Protecting individual rights and ensuring the safe and secure use of algorithms requires rules and governance for AI. Those building, procuring, and using AI have a responsibility to the people of Ontario to put appropriate guardrails in place before the technology is used. A risk-based approach will ensure public confidence in the use of AI technologies, ensuring potentially risky use cases adopt the needed rules to ensure safety and maintain individual rights.

Future success statement:

The use of AI in government services (such as application processing) is assessed for potential risks and any potential harm is mitigated by a robust framework of world-leading best practices.

Commitment 3: AI that serves all Ontarians

Problem

As AI-powered technologies continue to develop, they offer new ways to deliver better programs and services to Ontarians. That is why, as adoption increases, it is vitally important that the use of AI upholds democratic principles and individual rights and does not reinforce existing structures of discrimination, expand harmful surveillance, or threaten personal privacy. Therefore, it is important to require high standards from vendors and for people to have clear paths to challenge outcomes and improve the use of AI in decision-making.

Future success statement:

The government’s use of AI tools serves the public good and individuals have access to alternative processes should they want to use them.

Next steps

Thank you for sharing your ideas to improve the public’s trust in artificial intelligence (AI)

We will create and share actions under each commitment in Fall 2021 to meet our Open Government Partnership requirements.

These actions will form our AI Framework and we will continue to engage with you as we take our next steps towards trustworthy AI use in Ontario.