By Mitchell Pham, Chair of the Digital Council for Aotearoa New Zealand.
If the lifeblood of the digital economy is data, its heart is digital trust*. Trust, as we understand it from our research, is about whether people are comfortable in a situation where they are vulnerable to the consequences of someone else’s actions. We have been looking at trust in automated decision-making and what people need to be comfortable when the “someone else” is an algorithm.
Participants in our workshops have been clear that they want algorithms to be used alongside human involvement in decision making. But can we trust one over the other? Do we trust them both equally? How is each influenced by the other? And where does bias creep in — in the algorithm, in the human factor, or in the system intent that guides them?
The answer you get out depends on what you put in…
“An algorithm is just a fancy calculator,” says a participant from last week’s workshop with members of the disabled community. “Like, the answer you get out depends on what you put in — your criteria of the algorithm. So I absolutely trust the use of algorithms, but I don’t necessarily trust the criteria that are being fit into the algorithms.”
So, while on the surface some feel they will trust algorithms more when humans are involved in the process, there is recognition that humans bring and build in bias to the use of algorithms that may unfairly categorise, favour or lead decisions and outcomes.
Participants in last week’s workshop experienced this for themselves. When presented with a health scenario that used algorithms to determine who gets surgical treatment (something that could directly affect them), participants overall had lower trust in the algorithm. When presented with a scenario using algorithms to allow immigrants into New Zealand, they had higher trust in that algorithm — a participant suggested this was because it was something that wouldn’t affect them directly.
The closer it is to you, the more careful you are…
“It’s easier to be more dubious and take things more personally when we are personally involved,” said a participant. “But it’s different for deciding whether one of the six billion other people are to come to New Zealand. So the closer it is to you, the more careful you are, the lower the trust and the more dubious you are about the benefit.”
In all of our workshops, participants from Māori, Pasifika, migrant, youth, and disability communities have called for the right and necessity to be involved in creating algorithms that are used on their people. They talk about humans being at the centre of creating and utilising algorithms before, during and after the decision the algorithm makes. They already are, but the complexity of human input and human bias means the lifeblood of the digital economy requires diversity, inclusion and sovereignty to be at its heart.
ACTIVITY SUMMARY
- Last week we met with the Wellington City Council’s innovation lead, Sean Audain, who shared with us how local government is taking advantage of technology and data to improve citizens’ lives. We discussed the opportunities that central and local government have to improve digital inclusion and include citizens in policy and decision-making. We can see strong connections with this work as we think about our work programme in 2021, which is focused on inclusion.
- We also provided an overview of our role and the emerging insights we are hearing from our research at Te Mana Raraunga and Te Punaha Matatini’s Māori Data Sovereignty hui.
- Last year the Canadian Government released a directive on automated decision-making. We had a Zoom call with them last week to find out how that directive came about, how it is being implemented and what they’ve learned from the process.
*Quote from PWC Digital Insights 2019
