Connect with us

Society

Here’s a technique to assess the reliability of AI models before it’s deployed

“All models can be wrong, but models that know when they are wrong are more useful.”

Published

on

MIT Quantifying Reliability 01 press 0
Credits :MIT News; Courtesy of the researchers

MIT researchers, together with the MIT-IBM Watson AI Lab, have designed a method to assess the dependability of foundational AI models prior to their deployment in specific tasks.

Their method involves analyzing a set of slightly varied models and evaluating how consistently each model learns representations of the same test data point. Consistent representations indicate a reliable model.

This technique offers significant practical implications. It enables decision-makers to determine whether a AI model is suitable for deployment in specific settings without the necessity of testing it on real-world datasets

When compared against current advanced methods, their technique outperformed others in accurately measuring the reliability of foundation models across various downstream classification tasks.

This technique offers significant practical implications. It enables decision-makers to determine whether a AI model is suitable for deployment in specific settings without the necessity of testing it on real-world datasets. This capability is particularly valuable in contexts where accessing datasets is restricted due to privacy concerns, such as in healthcare settings. Furthermore, the method facilitates the ranking of models based on their reliability scores, empowering users to select the most appropriate model for their intended task.

Navid Azizan, senior author of the study and Esther and Harold E. Edgerton Assistant Professor in the MIT Department of Mechanical Engineering and the Institute for Data, Systems, and Society (IDSS), highlights the importance of models being aware of their own limitations: “All models can be wrong, but models that know when they are wrong are more useful.” He emphasizes the challenge in quantifying uncertainty or reliability for foundation models due to their abstract representations, which are inherently difficult to compare. Azizan concludes that their method offers a means to precisely quantify how reliable a model’s representation is for any given input data.

The research paper, authored by Young-Jin Park, a graduate student at LIDS; Hao Wang, a research scientist at the MIT-IBM Watson AI Lab; and Shervin Ardeshir, a senior research scientist at Netflix, will be presented at the Conference on Uncertainty in Artificial Intelligence.

Traditional machine-learning models are typically trained for specific tasks, providing definite predictions based on inputs such as determining whether an image features a cat or a dog. In contrast, foundation models are pretrained on generalized data without prior knowledge of the diverse tasks they will eventually tackle. Users fine-tune these models for specific applications after their initial training.

Unlike conventional models that yield straightforward outputs like “cat” or “dog,” foundation models generate abstract representations based on input data. Evaluating the reliability of such models poses a unique challenge. Researchers adopted an ensemble approach by training multiple models with similar characteristics but slight variations.

“Our concept is akin to gauging consensus. If all these foundation models consistently produce similar representations for any given dataset, then we can infer the model’s reliability,” explains Park.

The challenge arose in comparing these abstract representations. He says that these models output vectors, composed of numerical values, making direct comparisons difficult. The solution involved employing a method known as neighborhood consistency.

In their methodology, researchers established a set of dependable reference points to test across the ensemble of models. They examined neighboring points surrounding each model’s representation of the test data point to gauge consistency.

By assessing the coherence among neighboring points, they could effectively estimate the reliability of the models.

COP30

From 6% to 16%: The Philippines Shows the World How Fast Climate Budgets Can Shift

In just four years, the Philippines has expanded its climate spending from PHP 282 billion to over PHP 1 trillion — one of the fastest fiscal shifts anywhere in the world.

Published

on

Magazine 8
Flooded Street with Jeepney in Malabon, Philippines. Image credit: Tear Cordez/Pexels

Governments across the world are beginning to rethink the way national budgets are designed, moving away from traditional fiscal planning and toward systems that integrate climate considerations directly into spending decisions. A new comparative review of global green-budgeting practices reveals a trend that is gathering momentum: more countries are using their budgets as climate-governance tools. But the pace of progress varies sharply between advanced economies and emerging markets.

The Rise of Climate-Conscious Budgets

Countries such as France, Ireland, Mexico and the Philippines provide some of the clearest examples of how climate priorities are reshaping national expenditure. France has increased its identified climate-positive budget from €38.1 billion in 2021 to €42.6 billion in 2025, while Ireland expanded its environmental allocations from €2 billion (2020) to €7 billion (2025). Mexico’s transformation has been even more rapid: climate-related expenditures rose from MXN 70 billion (2021) to MXN 466 billion (2025) — a six-fold increase.

A Sudden Surge in the Philippines

Nowhere is the shift more dramatic than the Philippines. After embedding climate budget tagging across its ministries, the country’s climate budget expanded from PHP 282 billion in 2021 to more than PHP 1 trillion in 2025, raising its share of the national budget from 6% to 16%. The reform forced ministries to assess thousands of programmes through a climate lens, resulting in a shift toward resilient infrastructure, sustainable energy, water security, and climate-smart industries.

Advanced Economies Move Beyond Tagging

While emerging economies are scaling up climate allocations, advanced economies are integrating climate metrics deeper into fiscal systems. Canada’s “climate lens” requires greenhouse-gas and resilience assessments for major infrastructure projects before funding is approved. Norway links its annual budget to its Climate Change Act and long-term low-emission strategies. Germany uses sustainability indicators to guide fiscal decisions, embedding climate considerations into macroeconomic planning.

These tools go beyond transparency. They force ministries to justify public spending not only in economic terms, but in climate terms — shifting budgets from accounting documents to steering instruments.

Despite this momentum, the analysis notes a persistent gap: many countries stop at tagging climate-related expenditures without linking them to outcomes or performance indicators. Tagging improves transparency, but on its own does not change investment decisions. Without climate-based appraisal and monitoring, high-emission infrastructure can still slip through national budgets unchallenged.

The Financing Challenge

For lower-income countries, the largest barriers are financial. High capital costs, limited fiscal room, and weaker public financial management systems restrict the scale of green budgeting reforms. Even when climate spending rises, sustaining these increases requires integrating climate metrics into medium-term fiscal frameworks — something only a handful of emerging economies have attempted.

Innovations Show What’s Possible

Some models offer a blueprint. Indonesia’s climate-tagging system feeds directly into its sovereign green sukuk framework, giving investors clear visibility over the use of proceeds. This loop — tagging, reporting, financing — demonstrates how governments can leverage green budgeting to unlock larger pools of private capital.

Still in Progress

The report concludes that the next frontier for green budgeting is integration: linking budget tagging, climate-lens project appraisal, performance-based reporting, and climate-aligned fiscal strategies. Done together, these tools allow budgets to become climate-governance instruments capable of guiding national transitions.

But the pace remains uneven. Some countries are racing ahead, while others are taking incremental steps. What is clear, however, is that climate-aligned public finance is no longer optional. As climate impacts intensify, the alignment of the world’s budgets will determine who adapts — and who is left behind.

Continue Reading

COP30

Corporate Capture: Fossil Fuel Lobbyists at COP30 Hit Record High, Outnumbering Delegates from Climate-Vulnerable Nations

COP30 sees over 1,600 fossil fuel lobbyists inside climate talks, surpassing delegations of climate-vulnerable nations. Experts warn of corporate capture.

Published

on

cop jpeg
Illustrated image

COP30 was billed as the “Implementation COP,” a summit where governments would finally convert years of climate promises into concrete action. Instead, the year’s most striking headline comes from the corridors, not the negotiation rooms: more than 1,600 fossil fuel lobbyists have entered the talks — the highest in the history of the UN climate process.

A new analysis by the Kick Big Polluters Out (KBPO) coalition reveals that one in every 25 participants in Belém is linked to the oil, gas, or coal industry. The number surpasses the total delegations of many climate-vulnerable nations and even outnumbers the combined negotiating teams of the 10 most climate-impacted countries.

For many observers, the surge represents not just a statistic but a symptom of a deeper structural crisis.

“It’s common sense that you cannot solve a problem by giving power to those who caused it,” said Jax Bonbon of IBON International in a statement. “Yet three decades and 30 COPs later, more than 1,500 fossil fuel lobbyists are roaming the climate talks as if they belong here.”

A Climate Summit Outnumbered by Industry

The analysis shows 599 industry-linked representatives entered COP30 through Party overflow badges — a route typically reserved for government delegates. This method bypasses new transparency rules that require non-government participants to disclose their affiliations.

Untitled design 23
Illustration: S James/EdPublica

Several countries also included fossil fuel representatives directly within their official delegations. According to the report, France, Japan, and Norway brought senior industry figures, including those from TotalEnergies, Japan Petroleum Exploration, and Equinor.

“Until we Kick Big Polluters Out, we can expect the outcomes of COP30 — and every COP after — to be written by the world’s largest polluters,” said Pascoe Sabido of Corporate Europe Observatory. “It’s profit over people and the planet.”

The contrast between industry presence and the representation of climate-impacted nations is stark. The Philippines’ delegation is outnumbered by nearly 50 to 1. Jamaica sent fewer than 40 delegates — as it deals with the aftermath of Hurricane Melissa — while hundreds of industry lobbyists move freely inside the venue.

‘A Flood of Influence’

Civil society groups warn that the negotiations risk being shaped by the very actors accelerating the climate crisis.

“The COP is massively flooded with around 1,500 representatives of the fossil fuel industry — like a river bursting its banks and sweeping everything away,” said Susann Scherbarth of Friends of the Earth Germany.

The criticism echoes growing frustration among scientists and youth groups over the widening gap between climate science and political outcomes. Despite repeated warnings from the IPCC about the need for rapid fossil fuel phase-down, nearly $250 billion worth of new oil and gas projects have been approved since COP29.

Youth delegations expressed alarm that the negotiation space is becoming increasingly inaccessible to those most affected by the climate crisis.

“The UNFCCC is in need of rehabilitation,” said Pim Sullivan-Tailyour from the UK Youth Climate Coalition. “My generation deserves Just Transition policies shaped by what people and the planet need — not what polluters’ profits demand.”

Demands for Integrity and Accountability

Transparency and governance experts argue that the situation has reached a defining moment. “If COP30 is indeed the COP of truth, the Presidency and the UNFCCC Secretariat must strengthen participant disclosure rules,” said Brice Böhmer of Transparency International. “It is time to ensure integrity and restore trust.”

Civil society groups are urging governments to adopt formal conflict-of-interest rules, a step the UNFCCC has so far resisted. They argue that genuine climate progress requires insulating negotiations from actors whose core business models rely on continued fossil fuel extraction.

A Crossroads Moment for the UN Climate Process

COP30 was expected to accelerate global action toward limiting warming to 1.5°C. Instead, it has reopened a fundamental question: Can a climate summit deliver meaningful outcomes when the world’s largest polluters enjoy unprecedented access inside the process?

The KBPO coalition says the answer depends on whether the UNFCCC is willing to adopt structural reforms that prioritise vulnerable communities over powerful corporations.

As the talks continue in Belém, the tension between ambition and influence remains at the heart of COP30 — raising critical questions about transparency, accountability, and the future of global climate governance.

Continue Reading

Society

Guterres to WMO: ‘No Country Is Safe Without Early Warnings’

At WMO’s 75th anniversary, UN Chief António Guterres warned that no nation is safe from extreme weather — urging governments to fast-track early warning systems by 2027.

Joe Jacob

Published

on

UN CHIEF jpeg
Image credit: UN/Evan Schneider

Declaring that “no country is safe from the devastating impacts of extreme weather,” UN Secretary-General António Guterres called for a global surge in early warning systems to protect lives, economies, and ecosystems from climate-fuelled disasters.​

Speaking at the 75th anniversary of the World Meteorological Organization (WMO), Guterres hailed the agency as “a barometer of truth” and “a shining example of science supporting humanity.” It was his first address to the WMO, reflecting the agency’s central role in turning climate science into life-saving action.

“Without your rigorous modelling and forecasting, we would not know what lies ahead — or how to prepare for it,” he told delegates gathered at WMO headquarters in Geneva.

The occasion doubled as the midway checkpoint for the Early Warnings for All (EW4All) initiative, launched by Guterres in 2022 to ensure every person on Earth is protected by life-saving warning systems by 2027.

WMO Secretary-General Celeste Saulo issued a “Call to Action,” urging all countries to close early warning gaps through expanded observation networks, strengthened hydrological services, and community-level outreach. “Every dollar invested in early warning saves up to fifteen in disaster losses,” she said.​

Saulo cautioned that despite major progress—108 countries now operate multi-hazard warning systems—the world’s poorest remain the least protected. Disaster mortality rates are six times higher in countries with limited early warning coverage.​

A 75-Year Legacy of Science for Action

Marking 75 years since it became a UN specialized agency, WMO used its Extraordinary Congress to reaffirm global cooperation in weather, water, and climate monitoring.​

President Abdulla al Mandous praised Guterres for embedding early warning systems into the international climate agenda: “Early warnings are now recognized at the highest levels as cost-effective, life-saving, and cross-cutting solutions that reduce risk and advance development,” he said.

Guterres urged three urgent priorities to achieve universal coverage: integrating early warnings across governance structures, boosting finance and debt relief for vulnerable nations, and aligning national climate plans to limit temperature rise to 1.5°C.

“Every life lost to disaster is one too many,” he said. “With science, solidarity, and political resolve, we can ensure a safer planet for all.”

Continue Reading

Trending