An investigation into NSW’s debt collection agency found that it had abused its automated system to collect outstanding debts from vulnerable people. The state Ombudsman found that Revenue NSW’s use of artificial intelligence (AI) and machine learning was unlawful.
NSW Ombudsman Paul Miller calls for “greater transparency” and accountability for agencies relying on AI and machine learning technology to collect debts. During the state’s investigation it was found that Revenue NSW had negatively impacted vulnerable people with its automated debt-collection system.
According to the Ombudsman, Revenue NSW imposed automated garnishee orders that left some financially vulnerable people without the funds to eat or pay rent. Garnishee orders are used to debit money from bank accounts to settle debts. For those struggling financially, this can have dire consequences.
“My office began to receive a spate of complaints from people, many of them financially vulnerable individuals, who had discovered their bank accounts had been stripped of funds, and sometimes completely emptied,” said the Ombudsman in his report on the investigation. He also noted complainants were unaware of the automated process.
The Ombudsman acknowledges that Revenue NSW adjusted its process in 2019 in response to concerns raised, ensuring garnishee orders would no longer be completely automated. However, not all legal concerns were addressed by the changes.
“Following our intervention, and to its credit, Revenue NSW took a number of steps to address our concerns about unfairness,” said Mr Miller.
“However, what it did not do—despite our suggestion that it should—is seek expert legal advice on whether the use of the automation process was lawful and in accordance with its powers under the Fines Act. Ultimately, we decided to seek that advice ourselves.”
Going forward, the Ombudsman has a few key suggestions to ensure Revenue NSW ceases its harmful automated debt-collection processes.
The report suggests the need to redesign the process in consultation with experts from external fields, such as legal advisors. It is also recommended to develop “rigorous pre-deployment testing” and an “ongoing auditing regime”. The Ombudsman also recommends steps are taken to ensure and maintain transparency.
Finally, the report recommends that government-related agencies should seek Parliamentary approval before using machine technology for administrative uses that can have a significant impact on others.
“Seeking express legislative authorisation not only reduces the risk for agencies,” the Ombudsman said.
“It also gives Parliament and the public visibility of what is being proposed, and an opportunity to consider what other regulation of the technology may be required.”
It is hoped these steps can be taken to prevent future “maladministration”.
“We are concerned other agencies may also be designing and implementing machine technologies without appreciating all the risks, without transparency, and without getting appropriate legal advice,” said Mr Miller.
The NSW Government will now investigate and establish the dependence on machine technology across the state in administrative processes.
“Greater visibility is not a panacea to all of the potential issues that can arise when government adopts machine technology, but it is an essential starting point.”
