91���˰涶��

Using tech as a legal research tool can collate big data and provide an evidence base to inform policymaking, advocacy and legal reform.

New AI-driven legal research can help reduce barriers to justice for refugees and people seeking asylum, says an expert from UNSW Law & Justice. Associate Professor Daniel Ghezelbash is using computational methods to collate extensive data to build a much-needed evidence base to guide policymaking, institutional change and alternative interventions that would also benefit judges and other decision-makers.

“It's fair to say quantitative research has largely been lacking from the policy debate in the refugee space,” says the Deputy Director of UNSW’s Kaldor Centre for International Refugee Law and a practicing refugee lawyer. “We need to better leverage data on how asylum cases are processed and reviewed to improve both individual decision-making and institutional reform.”

Ensuring a fair process is vital now, as harsher, more restrictive policies take hold around the world, aimed at deterring people seeking asylum and blocking their access to territory, he says. This trend was only exacerbated in the COVID-19 pandemic. “We are witnessing a race to the bottom [in refugee policies] around the world,” he says.

“Technology can help us evaluate the fairness and efficiency of Australia’s processing of asylum claims; it can help detect potential systemic bias, develop systems of accountability and improve the lives of people seeking asylum in Australia.”

A multipronged approach

A/Prof. Ghezelbash specialises in international and comparative refugee and migration law. 

He is co-lead of the Access to Justice & Technology stream at the Allen’s Hub for Technology, Law and Innovation at UNSW. His interdisciplinary research works across law, computing, political science, international relations, behavioural psychology and data science, and is informed by his refugee background. 

“My parents came to Australia as refugees from Iran shortly before I was born. From a young age, I was aware of how lucky I was to be here and to enjoy the safety and opportunities available to me, compared to our friends and family back home,” he says. 

“I always felt with that privilege came a responsibility to assist others. I believe combining different perspectives and approaches offers the greatest potential to influence law and policy to deliver real-world outcomes for those in need of protection.”

Associate Professor Daniel Ghezelbash

In Australia, asylum seekers apply for protection visas through the Department of 91���˰涶�� Affairs. Those refused can seek review at the Administrative Appeals Tribunal (AAT) or at the Immigration Assessment Authority (IAA) if they arrived in Australia by boat without a valid visa. Those unsuccessful at the AAT or IAA can seek judicial review at the Federal Circuit and Family Court. 

A/Prof. Ghezelbash has conducted industry-first quantitative research into decision-making based on tens of thousands of reviews before the AAT, the Immigration Assessment Authority (IAA), and the Federal Circuit and Family Court. He plans to build on his current computational analysis with data-collection tools that leverage artificial intelligence (AI) capabilities. The research is one of the first international attempts to apply AI to aid our understanding of administrative or judicial decision-making in refugee case law.

The research draws on a combination of freedom of information requests and a programming script developed by his team to extract information from published cases. The data reveals vast discrepancies in protection visa outcomes across factors including legal representation and which judge or tribunal member was deciding the case. The research raises questions about equitable access to justice for people seeking asylum in Australia. 

“For the first time, we’re able to create a dashboard for every tribunal member and judge showing how many cases they find in favour of refugees and how many applications they refuse, and also to capture other data about the [refugee’s] country of origin and other factors that might affect the outcome,” he says.

What the numbers reveal

The research found success rates before different tribunal members varied greatly at the AAT, ranging from 0 per cent [decision-makers who never granted a visa] through to 89 per cent. While some discrepancies can be explained by the fact that case allocation is not random – some tribunal members specialise in cases from specific countries – such inconsistencies warrant further examination.

“We need to ensure institutions deliver fair, just, economical, informal and quick processes that promote public trust and confidence in their decision-making, according to their charters." 

Associate Professor Daniel Ghezelbash

“While you can’t distil the administrative and judicial system down to data alone, it provides unique insights to help us better understand issues of fairness and consistency in decision-making and refugee policies more broadly.” 

For example, between May 2020 and May 2022, refugees represented by a lawyer or migration agent at the AAT were nine times more likely to succeed than those without, the research found. 

“These statistics suggest the  to abolish public funding for free legal advice services [in 2013] may be severely disadvantaging applicants who cannot secure representation,” he says. “This is all the more concerning given our data show that more than half (52%) of all applicants do not have representation when they appear before the AAT.”

A/Prof. Ghezelbash is working towards a data-sharing arrangement with government to expand the data points covered. His project, funded by his Australian Research Council (ARC) Discovery Early Career Researcher Award (DECRA) grant, will also map national best practice to promote fair and efficient procedures for determining asylum claims. The research also provides a template for using technology and statistical regression analysis in empirical legal scholarship more broadly. 

Using data to improve decision-making

In December 2022, the Federal Government announced its intention to abolish the AAT, after a Senate inquiry identified issues with its productivity, integrity and performance, citing A/Prof. Ghezelbash’s research as evidence. A/Prof. Ghezelbash is briefing the Attorney General’s Office and the guiding the AAT reform; he is advocating for a data-driven approach to the reform process and better data collection and transparency within the new administrative review body.

“Data from public institutions should automatically be made publicly available. We need this data to identify the baseline for where we are now, and [we need to] build in data collection [as an operational imperative] to enable us to measure the impact of change,” he says.

The use of data by the courts was also addressed in the , released in August 2022. The report recommended Commonwealth courts “develop a policy on the creation, development, and use of statistical analysis of judicial decision-making”. The report extensively cited A/Prof. Ghezelbash’s research, in partnership with Macquarie University and the that evolved from the ‘Nudge Unit’ established by the UK government in 2010 to apply behavioural science to public policy. The Hon Justice Sarah C Derrington AM referenced the research as an example of how this recommendation could be implemented.

“It’s accepted now that judges, like everyone else, are subject to social and cognitive biases,” he says. “Using comparative data as a feedback tool is one of the only evidence-based interventions that has been shown to redress bias.” 

The effectiveness of this intervention lies in its promotion of analytic (slower) decision-making, rather than intuitive decision-making where social and cognitive biases come into play, encouraging us to identify and overcome our unconscious bias, he says.

“[Comparing] data provides an opportunity for the head of the jurisdiction to have a conversation with a judge or tribunal member … to reflect on explanations …. [for why] they are diverging so much from their peers deciding similar cases,” he says.

“The evidence [also] suggests it’s even more effective to make that data public. And so that's what we've done with the Kaldor Centre Data Lab. Greater data transparency is imperative to maintaining a fair and effective justice system.”

A/Prof. Ghezelbash created the in 2022 to regularly publish and analyse data to better understand Australia’s refugee status determination procedures. 

The Data Lab also plays an important role in democratising access to jurimetrics, with its data used widely by lawyers to inform their litigation strategy, he says. “Anecdotally, everyone knows that one of the biggest factors that influences a client’s chances of success is which decision maker you get,” he says. “The Data Lab quantifies that.”

Before the Data Lab, insights into judges’ decision-making were available only to large commercial law firms who can subscribe to expensive proprietary tools. “So it’s not only [a case of] whether you can afford an expensive lawyer or not, but also whether you can afford these high-tech tools that give you an extra edge in terms of how lawyers craft their legal arguments,” A/Prof. Ghezelbash says.

“The risk is that this will further exacerbate the access-to-justice gap. The Kaldor Centre Data Lab aims to level the field and ensure this data is freely available to all.”

Written by Kay Harrison

Researcher