Connect with us

Society

Here’s a technique to assess the reliability of AI models before it’s deployed

“All models can be wrong, but models that know when they are wrong are more useful.”

Published

on

Credits :MIT News; Courtesy of the researchers

MIT researchers, together with the MIT-IBM Watson AI Lab, have designed a method to assess the dependability of foundational AI models prior to their deployment in specific tasks.

Their method involves analyzing a set of slightly varied models and evaluating how consistently each model learns representations of the same test data point. Consistent representations indicate a reliable model.

This technique offers significant practical implications. It enables decision-makers to determine whether a AI model is suitable for deployment in specific settings without the necessity of testing it on real-world datasets

When compared against current advanced methods, their technique outperformed others in accurately measuring the reliability of foundation models across various downstream classification tasks.

This technique offers significant practical implications. It enables decision-makers to determine whether a AI model is suitable for deployment in specific settings without the necessity of testing it on real-world datasets. This capability is particularly valuable in contexts where accessing datasets is restricted due to privacy concerns, such as in healthcare settings. Furthermore, the method facilitates the ranking of models based on their reliability scores, empowering users to select the most appropriate model for their intended task.

Navid Azizan, senior author of the study and Esther and Harold E. Edgerton Assistant Professor in the MIT Department of Mechanical Engineering and the Institute for Data, Systems, and Society (IDSS), highlights the importance of models being aware of their own limitations: “All models can be wrong, but models that know when they are wrong are more useful.” He emphasizes the challenge in quantifying uncertainty or reliability for foundation models due to their abstract representations, which are inherently difficult to compare. Azizan concludes that their method offers a means to precisely quantify how reliable a model’s representation is for any given input data.

The research paper, authored by Young-Jin Park, a graduate student at LIDS; Hao Wang, a research scientist at the MIT-IBM Watson AI Lab; and Shervin Ardeshir, a senior research scientist at Netflix, will be presented at the Conference on Uncertainty in Artificial Intelligence.

Traditional machine-learning models are typically trained for specific tasks, providing definite predictions based on inputs such as determining whether an image features a cat or a dog. In contrast, foundation models are pretrained on generalized data without prior knowledge of the diverse tasks they will eventually tackle. Users fine-tune these models for specific applications after their initial training.

Unlike conventional models that yield straightforward outputs like “cat” or “dog,” foundation models generate abstract representations based on input data. Evaluating the reliability of such models poses a unique challenge. Researchers adopted an ensemble approach by training multiple models with similar characteristics but slight variations.

“Our concept is akin to gauging consensus. If all these foundation models consistently produce similar representations for any given dataset, then we can infer the model’s reliability,” explains Park.

The challenge arose in comparing these abstract representations. He says that these models output vectors, composed of numerical values, making direct comparisons difficult. The solution involved employing a method known as neighborhood consistency.

In their methodology, researchers established a set of dependable reference points to test across the ensemble of models. They examined neighboring points surrounding each model’s representation of the test data point to gauge consistency.

By assessing the coherence among neighboring points, they could effectively estimate the reliability of the models.

Society

INM: MIT’s Bold Push to Regain America’s Productive Edge

The ambitious initiative aims at reinvigorating U.S. manufacturing with cutting-edge innovation

Published

on

MIT President Sally A. Kornbluth. Image credit: Jake Belcher/MIT

In a move to reshape the future of American industry, the Massachusetts Institute of Technology (MIT) has launched its Initiative for New Manufacturing (INM), an Institute-wide effort aimed at revitalizing U.S. manufacturing through next-generation technologies, research, education, and deep collaboration with industry.

Announced today, INM seeks to strengthen key sectors of the U.S. economy and spark nationwide job creation. The initiative will bring together MIT’s extensive research capabilities and educational resources to help companies of all sizes increase productivity and build a more resilient and human-centered manufacturing landscape.

“We want to work with firms big and small, in cities, small towns and everywhere in between, to help them adopt new approaches for increased productivity,” MIT President Sally A. Kornbluth wrote in a letter to the Institute community this morning. “We want to deliberately design high-quality, human-centered manufacturing jobs that bring new life to communities across the country.”

“We want to work with firms big and small, in cities, small towns and everywhere in between, to help them adopt new approaches for increased productivity

Kornbluth emphasized the significance of the effort, stating in a media statement: “Helping America build a future of new manufacturing is a perfect job for MIT — and I’m convinced that there is no more important work we can do to meet the moment and serve the nation now.”

Industry Collaboration

INM has already attracted strong industry support, with its first five founding consortium members — Amgen, GE Vernova, PTC, Siemens, and Sanofi — joining forces to fund initial research projects, particularly in the area of artificial intelligence for manufacturing.

“There is tremendous opportunity to bring together a vibrant community working across every scale — from nanotechnology to large-scale manufacturing,” said Anantha Chandrakasan, MIT’s chief innovation and strategy officer and dean of engineering. “MIT is uniquely positioned to harness the transformative power of digital tools and AI to shape the future of manufacturing.”

The initiative will support research, education, and real-world applications — including new manufacturing labs, a “factory observatory” program to connect students with live production sites, and thematic pillars ranging from semiconductors and biomanufacturing to defense and aviation.

Workforce development is also central to INM’s mission. It will include TechAMP, a program designed to bridge the gap between technicians and engineers through collaboration with community colleges, along with AI-powered teaching tools and expanded manufacturing education on campus.

The initiative is co-directed by three MIT faculty: John Hart, head of mechanical engineering; Suzanne Berger, an Institute Professor and political scientist; and Chris Love, professor of chemical engineering. Julie Diop serves as executive director.

At a recent MIT symposium titled “A Vision for New Manufacturing,” Berger underscored the urgency of the moment: “The rationale for growing and transforming U.S. manufacturing has never been more urgent than it is today. What we are trying to build at MIT now is not just another research project. … Together, with people in this room and outside this room, we’re trying to change what’s happening in our country.”

Love added: “We need to think about the importance of manufacturing again, because it is what brings product ideas to people… There is a real urgency about this issue for both economic prosperity and creating jobs.”

Echoing the sentiment, Hart emphasized the long-term significance of the initiative: “While manufacturing feels very timely today, it is of enduring importance… Working with industry — from small to large companies, and from young startups to industrial giants — will be instrumental to creating impact and realizing the vision for new manufacturing.”

A Continuum of Commitment

INM builds on a legacy of MIT initiatives aimed at supporting manufacturing, including the 1989 book Made in America, the Production in the Innovation Economy project, and The Engine, a venture fund launched in 2016 to back hardware-based startups.

As Kornbluth noted in her letter, “We want to reimagine manufacturing technologies and systems to advance fields like energy production, health care, computing, transportation, consumer products, and more… and we want to reach well beyond the shop floor to tackle challenges like how to make supply chains more resilient, and how to inform public policy to foster a broad, healthy manufacturing ecosystem that can drive decades of innovation and growth.”

With its launch, MIT’s Initiative for New Manufacturing marks a renewed commitment to restoring American manufacturing leadership through innovation, collaboration, and education — aimed squarely at building a stronger, more equitable industrial future.

Continue Reading

EDUNEWS & VIEWS

Harvard Pledges $250 Million for Research After Federal Funding Slash

The administration has defended the funding freeze as part of a broader campaign to address what it characterizes as pervasive anti-Semitism on campuses and to roll back diversity programs

Published

on

Image credit: Kalhh from Pixabay

Harvard University has announced a $250 million investment to sustain vital research programs in the face of steep federal funding cuts imposed by the Trump administration.
The move follows a sweeping $2.6 billion reduction in government grants to the Ivy League institution, citing alleged discriminatory practices and refusal to comply with federal oversight mandates. The cuts, which Harvard is actively challenging in court, have already suspended or canceled dozens of projects—some of which were considered critical to public health and technological innovation.

University President Alan Garber and Provost John Manning issued a joint statement on Wednesday, emphasizing the urgent need to protect research initiatives. “While we cannot fully offset the financial blow from halted federal support, we are committed to backing essential research during this transitional period,” they said. The university is also working with faculty to secure alternative funding channels.

Harvard has strongly criticized the federal measures, calling the termination of grants “unlawful” and accusing the administration of interfering with academic independence. The university contends that the loss of funding not only halts groundbreaking work but also threatens years of scientific progress.

At the heart of the dispute is a broader political clash over university governance. Harvard, whose endowment reached $53.2 billion in 2024, has become a focal point of the Trump administration’s efforts to reshape higher education policy. The White House has demanded greater control over admissions, hiring, and the political climate on campus—demands Harvard has resisted.

The administration has defended the funding freeze as part of a broader campaign to address what it characterizes as pervasive anti-Semitism on campuses and to roll back diversity programs. Critics argue these moves are part of a larger effort to suppress progressive academic culture and penalize dissent over U.S. foreign policy, especially in light of recent student protests against the war in Gaza.

In recent weeks, federal authorities have also taken steps to revoke visas of international students involved in these demonstrations, accusing them of ties to militant organizations—allegations civil rights groups and university leaders have strongly disputed.

With tensions between the federal government and top academic institutions mounting, Harvard’s legal challenge could set a precedent for how universities navigate political interference while safeguarding research, free speech, and academic autonomy.

Continue Reading

Health

Robot Helps Elderly Sit, Stand, and Stay Safe from Falls

The innovation comes at a time when the United States faces a dramatic demographic shift

Published

on

Image credit: MIT News/ Courtesy of the researchers

As America’s population ages faster than ever before, a team of engineers at MIT is turning to robotics to meet the growing eldercare crisis. Their latest invention, the Elderly Bodily Assistance Robot—or E-BAR—aims to provide critical physical support to seniors navigating life at home, potentially reducing the risk of injury and relieving pressure on a strained care system.

The innovation comes at a time when the United States faces a dramatic demographic shift. The nation’s median age has climbed to 38.9, nearly ten years older than in 1980. By 2050, the number of adults over 65 is projected to surge from 58 million to 82 million. As demand for care rises, the country is simultaneously grappling with shortages in care workers, escalating healthcare costs, and evolving family structures that leave many elderly adults without daily support.

“Eldercare is the next great challenge,” said Roberto Bolli, a graduate student in MIT’s Department of Mechanical Engineering and one of E-BAR’s lead designers, in a media statement. “All the demographic trends point to a shortage of caregivers, a surplus of elderly persons, and a strong desire for elderly persons to age in place.”

E-BAR is designed to address exactly that challenge. The mobile robot acts as a robotic support system, following a user from behind and offering both steadying handlebars and rapid intervention in case of a fall. It can support a person’s full weight and includes side airbags that inflate instantly to catch users if they begin to fall—without requiring them to wear any equipment or harnesses.

“Many older adults underestimate the risk of fall and refuse to use physical aids, which are cumbersome, while others overestimate the risk and may not exercise, leading to declining mobility,” said Harry Asada, the Ford Professor of Engineering at MIT, in a media statement. “Our design concept is to provide older adults having balance impairment with robotic handlebars for stabilizing their body. The handlebars go anywhere and provide support anytime, whenever they need.”

The robot consists of a heavy, 220-pound base equipped with omnidirectional wheels, allowing it to maneuver easily through typical home spaces. From its base, articulated bars extend and adjust to assist users in standing or sitting, and the handlebars provide a natural, unrestrictive grip. In testing, E-BAR successfully helped an older adult complete everyday movements such as bending, reaching, and even stepping over the edge of a bathtub.

“Seeing the technology used in real-life scenarios is really exciting,” said Bolli.

The team’s design, which will be presented later this month at the IEEE Conference on Robotics and Automation (ICRA), aims to eliminate the physical constraints and stigmas often associated with eldercare devices. Their approach prioritizes both independence and safety—key values for aging Americans seeking to remain in their homes longer.

While E-BAR currently operates via remote control, the team plans to add autonomous capabilities and streamline the device’s design for home and facility use. The researchers are also exploring ways to integrate fall-prediction algorithms, developed in a parallel project in Asada’s lab, to adapt robotic responses based on a user’s real-time risk level.

“Eldercare conditions can change every few weeks or months,” Asada noted. “We’d like to provide continuous and seamless support as a person’s disability or mobility changes with age.”

As the nation prepares for the realities of an aging population, MIT’s work offers a glimpse into a future where robotics play a central role in eldercare—enhancing both quality of life and personal dignity for millions of older adults.

Continue Reading

Trending