21 janvier 2020 | International, Méga données et intelligence artificielle

Trustworthy AI: A Conversation with NIST's Chuck Romine

Trustworthy AI: A Conversation with NIST's Chuck Romine

By: Charles Romine

Artificial Intelligence (AI) promises to grow the economy and improve our lives, but with these benefits, it also brings new risks that society is grappling with. How can we be sure this new technology is not just innovative and helpful, but also trustworthy, unbiased, and resilient in the face of attack? We sat down with NIST Information Technology Lab Director Chuck Romine to learn how measurement science can help provide answers.

How would you define artificial intelligence? How is it different from regular computing?

One of the challenges with defining artificial intelligence is that if you put 10 people in a room, you get 11 different definitions. It's a moving target. We haven't converged yet on exactly what the definition is, but I think NIST can play an important role here. What we can't do, and what we never do, is go off in a room and think deep thoughts and say we have the definition. We engage the community.

That said, we're using a narrow working definition specifically for the satisfaction of the Executive Order on Maintaining American Leadership in Artificial Intelligence, which makes us responsible for providing guidance to the federal government on how it should engage in the standards arena for AI. We acknowledge that there are multiple definitions out there, but from our perspective, an AI system is one that exhibits reasoning and performs some sort of automated decision-making without the interference of a human.

There's a lot of talk at NIST about “trustworthy” AI. What is trustworthy AI? Why do we need AI systems to be trustworthy?

AI systems will need to exhibit characteristics like resilience, security and privacy if they're going to be useful and people can adopt them without fear. That's what we mean by trustworthy. Our aim is to help ensure these desirable characteristics. We want systems that are capable of either combating cybersecurity attacks, or, perhaps more importantly, at least recognizing when they are being attacked. We need to protect people's privacy. If systems are going to operate in life-or-death type of environments, whether it's in medicine or transportation, people need to be able to trust AI will make the right decisions and not jeopardize their health or well-being.

Resilience is important. An artificial intelligence system needs to be able to fail gracefully. For example, let's say you train an artificial intelligence system to operate in a certain environment. Well, what if the system is taken out of its comfort zone, so to speak? One very real possibility is catastrophic failure. That's clearly not desirable, especially if you have the AI deployed in systems that operate critical infrastructure or our transportation systems. So, if the AI is outside of the boundaries of its nominal operating environment, can it fail in such a way that it doesn't cause a disaster, and can it recover from that in a way that allows it to continue to operate? These are the characteristics that we're looking for in a trustworthy artificial intelligence system.

NIST is supposed to be helping industry before they even know they needed us to. What are we thinking about in this area that is beyond the present state of development of AI?

Industry has a remarkable ability to innovate and to provide new capabilities that people don't even realize that they need or want. And they're doing that now in the AI consumer space. What they don't often do is to combine that push to market with deep thought about how to measure characteristics that are going to be important in the future. And we're talking about, again, privacy, security and resilience ... trustworthiness. Those things are critically important, but many companies that are developing and marketing new AI capabilities and products may not have taken those characteristics into consideration. Ultimately, I think there's a risk of a consumer backlash where people may start saying these things are too easy to compromise and they're betraying too much of my personal information, so get them out of my house.

What we can do to help, and the reason that we've prioritized trustworthy AI, is we can provide that foundational work that people in the consumer space need to manage those risks overall. And I think that the drumbeat for that will get increasingly louder as AI systems begin to be marketed for more than entertainment. Especially at the point when they start to operate critical infrastructure, we're going to need a little more assurance.

That's where NIST can come together with industry to think about those things, and we've already had some conversations with industry about what trustworthy AI means and how we can get there.

I'm often asked, how is it even possible to influence a trillion-dollar, multitrillion-dollar industry on a budget of $150 million? And the answer is, if we were sitting in our offices doing our own work independent of industry, we would never be able to. But that's not what we do. We can work in partnership with industry, and we do that routinely. And they trust us, they're thrilled when we show up, and they're eager to work with us.

AI is a scary idea for some people. They've seen “I, Robot,” or “The Matrix,” or “The Terminator.” What would you say to help them allay these fears?

I think some of this has been overhyped. At the same time, I think it's important to acknowledge that risks are there, and that they can be pretty high if they're not managed ahead of time. For the foreseeable future, however, these systems are going to be too fragile and too dependent on us to worry about them taking over. I think the biggest revolution is not AI taking over, but AI augmenting human intelligence.

We're seeing examples of that now, for instance, in the area of face recognition. The algorithms for face recognition have improved at an astonishing rate over the last seven years. We're now at the point where, under controlled circumstances, the best artificial intelligence algorithms perform on par with the best human face recognizers. A fascinating thing we learned recently, and published in a report, is that if you take two trained human face recognizers and put them together, the dual system doesn't perform appreciably better than either one of them alone. If you take two top-performing algorithms, the combination of the two doesn't really perform much better than either one of them alone. But if you put the best algorithm together with a trained recognizer, that system performs substantially better than either one of them alone. So, I think, human augmentation by AI is going to be the revolution.

What's next?

I think one of the things that is going to be necessary for us is pulling out the desirable characteristics like usability, interoperability, resilience, security, privacy and all the things that will require a certain amount of care to build into the systems, and get innovators to start incorporating them. Guidance and standards can help to do that.

Last year, we published our plan for how the federal government should engage in the AI standards development process. I think there's general agreement that guidance will be needed for interoperability, security, reliability, robustness, these characteristics that we want AI systems to exhibit if they're going to be trusted.

https://www.nist.gov/blogs/taking-measure/trustworthy-ai-conversation-nists-chuck-romine

Sur le même sujet

  • Boeing Expected To Follow Airbus Production Cuts

    17 avril 2020

    Boeing Expected To Follow Airbus Production Cuts

    Guy Norris Jens Flottau Michael Bruno Sean Broderick April 17, 2020 Aviation Week and Space technology Just a short time ago, Airbus could not expand fast enough. Given the strength of demand, the manufacturer planned to add another final assembly line for narrowbodies in Toulouse, even though its global industrial system was already complex — because maximum deliveries were what mattered. Now, as aircraft manufacturers begin to assess the medium- and longer-term impact of the COVID-19 pandemic, the Toulouse project and many others- are on hold or at least substantially slowed down. And if the revised projections for future aircraft deliveries are only roughly accurate, that additional assembly line will not be needed for a very long time. Airbus in early April became the first OEM to announce new production targets, around one-third below previous assumptions. At this stage, trying to figure out aircraft demand and future production rates is a matter of likelihoods and scenarios covering a wide range of outcomes. In many cases, the most optimistic scenarios already look outdated, leaving those indicating a deeper and more lasting impact on the table — a grim outlook for Airbus, Boeing, Embraer and their suppliers. Consultancy Roland Berger has attempted to define three possible outcomes. Its “rebound” scenario is modeled around two months of air travel restrictions, a full recovery to a precrisis air travel level by next winter and a compound annual growth rate (CAGR) of 4.6% thereafter. In that case, OEMs would lose only 790 deliveries over 10 years vis-a-vis the precrisis outlook. It is already clear that the short term will be much worse, although a steep recovery in later years remains a possibility. If restrictions stay in place for four months, the new normal would be at 90% of precrisis levels and would be reached in the summer of 2021. Airlines would defer aircraft replacement for 18 months and future growth rates average 4.1%. Under this “delayed cure” scenario, the industry would lose almost 6,000 deliveries between now and 2030. Unfortunately, the worst Roland Berger scenario may now be the most likely, at least in its short-term elements: Six months of severe travel restrictions, demand recovery to only 80% by the summer of 2022, extended deferrals and lower growth for the long term at 3.6%. In that circumstance, airlines would accept 10,460 fewer aircraft over the next 10 years. For context: Airbus delivered 863 aircraft in 2019, and Boeing sent out 380 (affected by the 737 MAX grounding). In 2019, Boeing led with 806 deliveries, and Airbus handed over 800 aircraft. If the worst-case scenario comes to pass, the industry will deliver only 11,280 aircraft in the next 10 years; the more positive assumptions of the “delayed cure” model leave that number at 15,840. This would essentially put the industry—on average and very roughly—at the 2017 delivery levels of a combined 1,600 commercial jets above 100 seats for Airbus (now including the A220), Boeing and Embraer. Of course, new players such as Mitsubishi's SpaceJet, the Comac C919 and the United Aircraft Corp. MC-21 will fill (small) amounts of the demand as well. Airbus' decision to cut production by one-third would, if continued, leave it with around 570 annual deliveries, close to 2011 levels. CEO Guillaume Faury points out that it is “not unlikely” that the new rates could go back up in 2021 as the situation improves, but he says it is too early to make firm commitments. The decisions made now reflect the “best knowledge” today and “many conversations with airline CEOs and [chief operating officers].” The production reduction will be implemented over the coming weeks. “This crisis will probably be a long one,” says Faury. “Our industry is one of the most impacted. . . . [The production rates are] the result of the best matching between the downturn and the remaining commitments. We needed to have a plan. We will review it probably on a monthly basis.” The manufacturer plans to produce 40 A320neo-family aircraft per month on average, down from a previous near-term target of 63 and a planned increase to 67 or more. Airbus will also reduce A330/A330neo output to two aircraft per month from the precrisis production level of 3.5. A350 deliveries are being reduced from 9-10 aircraft per month to just six. Airbus delivered 122 aircraft in the first quarter, 36 of them in March. That month included two A220s, 19 A320neos, one A321ceo, 10 A321neos, one A330-200 and three A350s, according to the company's latest order and delivery figures. Airbus produced 60 more aircraft that were not delivered in the quarter because customers said they were unable to accept them. The A330 will remain a profitable program, says Faury, but the A350 will face “more headwinds,” having just moved into profitability in 2019. Boeing's recently updated commercial airliner figures for 2020 through March reveal dramatic cuts in orders and deliveries as the air transport market continues to nosedive in the midst of the COVID-19 pandemic. The beleaguered company, which is due to release its first-quarter financial results on April 29, saw net orders for the year plummet by 307 aircraft, putting it on track for its worst period since the mid-1990s. Despite picking up 24 new orders for the 787 and two additional 767 orders, the bulk of the damage was caused by losses to the 737 MAX orderbook. According to many financial analysts, Airbus' rate cuts set a floor for similar action by Boeing. “We believe similar cuts from Boeing are likely,” Ken Herbert of Canaccord Genuity said April 14. His team assumes MAX deliveries will not restart until at least the third quarter, with just 36 to be delivered this year. It will take “several quarters” for MAX new order activity to pick up. Production next year could average 21 new narrowbodies monthly, and perhaps around 40 per month by the end of 2022. But some of that depends on how much government aid and commercial-sector funding Boeing ultimately receives. “We believe the updated production forecasts are alarming, but not surprising,” Herbert added. “The coronavirus is likely to become a significantly greater pressure point on Boeing than the long-running 737 MAX crisis,” says Jonathan Root, Moody's Investors Service senior vice president and lead analyst. “We now estimate external funding needs in 2020 to at least double—to $30 billion—compared to our precoronavirus expectations,” he says. Boeing already funded about half of this need with the $13.8 billion delayed-draw term loan facility arranged in February and drawn down by mid-March. In the long term, Root's team does not envisage a return to 2019 delivery numbers before the end of 2022. Many other analysts such as Herbert and consultants such as Roland Berger's group agree. Grounded since March 2019 following two accidents, the 737 MAX lost a further 150 orders in March 2020, half of which were cancellations from aircraft lessor Avolon. Other operators included Brazilian carrier Gol, which cut 34 aircraft from the backlog as part of a compensation deal for nondelivery of 25 MAXs in 2019. Overall, net 737 orders for the first quarter have been reduced by 314 aircraft, some 173 of which are listed as cancellations or conversions to other models, and another 141 lost because they no longer meet Boeing's firm contract revenue accounting standard. Deliveries were also significantly down. Just 50 aircraft of all models were delivered through March 31, representing the lowest number of quarterly deliveries since the end of 2008. By comparison, Boeing delivered 149 aircraft in the same period in 2019, and 184 were handed over to customers in the first quarter of 2018. The falloff in deliveries primarily reflects the continuing impact of the 737 MAX grounding, which last year more than halved the company's overall delivery target. Boeing says delivery numbers have been also affected by logistical challenges, as some operators have been unable to bring flight crews to the U.S. to accept the new aircraft owing to travel restrictions. The restrictions are also hampering the MAX's return-to-service effort. The final FAA certification flight to put the software through its paces, the next major step in the return-to-service process, has not taken place. The subsequent milestone, work by the Joint Operational Evaluation Board (JOEB), a multiregulatory group that will evaluate the MAX training recommendations in simulator sessions, has not been scheduled and is not likely to happen until international travel restrictions are eased. The MAX will not be approved for service until the JOEB's work, including a written report, is done. Boeing has not wavered from its projection that a mid-2020 FAA approval is in the cards, but the lack of progress on key milestones makes this increasingly unlikely. In 2019, Boeing completed the year with 380 deliveries, 127 of which were 737s and 158 787s. The previous year, with production of the MAX ramping up and manufacture of the final 737NGs still being phased out, the single-aisle models accounted for 580 of the record-breaking 806 deliveries the company racked up for the year. Military deliveries accounted for eight of the 50 (five KC-46A/767-derivative tankers and three P-8A maritime patrol 737NG derivatives), while the 787 contributed to the bulk of the tally with 29 aircraft. Boeing has meanwhile announced it will begin a phased return to production of commercial aircraft models—including the 737MAX—at its Puget Sound, Washington, and South Carolina plants starting as early as April 20 after suspending activity on March 25 due to the COVID-19 outbreak. It is, however, widely expected to announce rate reductions for the 737, 777 and 787 shortly after manufacturing resumes. The adjusted numbers, using Boeing's own accounting standards on firm contracts, now show the overall firm backlog for all models has been reduced to 5,049—4,079 of those are for the 737. The 747-8F backlog is now reduced to 13, while the 777 orderbook has shrunk to 356 and the 787 to 515. Business aviation fares no better than civil, with an expectation that the business jet market will see deteriorating demand. “Though we expect demand for smaller and midsize jets will see a greater decline in demand than larger jets, there will still be a negative impact to the large-cabin segment that includes Bombardier's Global family,” Moody's analysts say. In turn, credit rating agencies such as Moody's, S&P Global Ratings and Fitch Ratings are downgrading debt rankings of OEMs and suppliers across the board. “The downgrades reflect Moody's expectation that 2020 will be a very challenging year for commercial aerospace suppliers, with double--digit earnings declines stemming from a significant reduction in commercial aerospace production by Tier 1 OEMs and suppliers,” the Moody's analysts say. “Stress on the supply chain will result in unprecedented deterioration in earnings and cash flows, resulting in key credit metrics that will remain strained for some time.” “Many suppliers are distressed,” Spirit AeroSystems said in a regulatory filing.

  • Aviation Startups Making Progress, But Can They Disrupt The Industry?

    3 décembre 2020

    Aviation Startups Making Progress, But Can They Disrupt The Industry?

    Graham Warwick December 02, 2020 The concept of a minimum viable product is not new to aviation. It is how the industry started. But as aircraft technology has advanced, customers have come to expect more than a minimum capability. Along comes Silicon Valley's startup culture, with its drive to find a foothold from which to launch a new technology—a less-than-perfected product that can be developed quickly to disrupt or create a market. How well is that going for aviation? From autonomy and artificial intelligence (AI) to hybrid-electric and hydrogen propulsion, is there a viable product taking shape that can perform a valuable mission? Autonomy The vision: unmanned cargo aircraft plying the skies to meet the ever-growing express logistics needs of the e-commerce giants. The reality: a pair of startups that are converting the Cessna Caravan into a remotely piloted regional cargo aircraft as a first step. The goal is that supervised autonomy would enable several aircraft to be managed by one remote pilot on the ground, increasing aircraft utilization and reducing operating costs. Reliable Robotics and Xwing plan to operate their aircraft manned initially, the autonomy advising the pilot while accumulating the experience required to certify the system. The companies hope to begin commercial flights by 2022. There are plenty of startups pursuing the express logistics market with unmanned cargo aircraft, but by targeting an existing market—several hundred Caravans fly as freight feeders for package carriers—and modifying an already certified aircraft and taking a staged approach to introducing autonomy, these two companies hope to lower the certification hurdles. Artificial Intelligence The vision: automated aircraft flown by machine-learning algorithms that replicate the skills of human pilots but not their mistakes. The reality: The initial approach is to use AI to help the pilot in high-workload phases of flight, such as landing. Swiss startup Daedalean is developing a camera-based system to provide safe landing guidance for general-aviation aircraft and vertical-takeoff-and-landing vehicles. Airbus has the longer-term goal of bringing autonomy to its commercial aircraft but has started in the same place, demonstrating fully automatic vision-based takeoffs and landings with an A350 in April. By tackling one well-defined subtask of visual flying, and proving the system can be safer than human piloting, Daedalean hopes to create the path to certification of AI for safety-critical applications. The European Union Aviation Safety Agency, which has been working with the startup to frame the rules, expects the first AI applications to be certified in 2022. Hybrid-Electric The vision: propulsion systems that overcome the limitations of batteries to deliver the economic and emissions benefits of electrification in larger, faster, longer-range aircraft. The reality: Starting small, startups Ampaire and VoltAero are testing power trains in converted Cessna 337 Skymasters. Ampaire's route to market is to modify existing aircraft, beginning with the Skymaster as the four-seat, 200-mi. Electric EEL but moving on to the 19-seat de Havilland Canada Twin Otter. France's VoltAero, meanwhile, is taking the clean-sheet approach with plans for a family of hybrid-electric aircraft with up to 10 seats and 800-mi. range. Delivery of the initial four-seat Cassio 330 version is planned for 2023. While batteries have improved enough to make pure-electric urban air taxis feasible, longer ranges are still out of reach. But there are startups working to field all-electric nine- and 19-seat aircraft within just a couple of years of the first hybrid-electric types. It remains to be seen whether hybrid propulsion is just a stopgap, as with cars, or a long-term market niche. Hydrogen The vision: zero-emissions flight for aircraft of all sizes and ranges. The reality: adapting automotive fuel-cell technology to modify regional turboprops and kick-start the market for green hydrogen as an aviation fuel. ZeroAvia made the first flight of a six-seater with a fuel-cell power train from Cranfield, England, in September and plans a 300-mi. demonstration flight. The startup's route to market is to modify existing 10- and 20-seaters to hydrogen-electric propulsion, aiming for its first certification within three years. Universal Hydrogen is more ambitious, targeting the 50-seat de Havilland Canada Dash 8-300 for conversion to hydrogen fuel-cell propulsion for market entry by 2024. Introducing a new fuel to aviation is an infrastructure issue. By starting small, the startups believe the challenge of producing green hydrogen can be made manageable. But to have an impact on aviation's contribution to climate change, hydrogen needs to be scaled up to larger and larger aircraft as quickly as possible. https://aviationweek.com/aerospace/emerging-technologies/aviation-startups-making-progress-can-they-disrupt-industry

  • How DoD can improve its technology resilience

    17 décembre 2020

    How DoD can improve its technology resilience

    Mark Pomerleau WASHINGTON — The Department of Defense must bolster its resilience in mission platforms in order to stay ahead of threats, a new think tank report says. With the military's shift toward great power competition, or conflict against nation states, its systems and platforms will be under greater stress than technological inferior adversaries battled during the counterterrorism fight of the last decade-plus. Systems and networks are expected to be contested, disrupted and even destroyed, meaning officials need to build redundancy and resilience in from the start to work through such challenges. In fact, top defense officials have been warning for several years that they are engaged in conflict that is taking place below the threshold of armed conflict in which adversaries are probing networks and systems daily for espionage or disruptive purposes. “Resilience is a key challenge for combat mission systems in the defense community as a result of accumulating technical debt, outdated procurement frameworks, and a recurring failure to prioritize learning over compliance. The result is brittle technology systems and organizations strained to the point of compromising basic mission functions in the face of changing technology and evolving threats,” said a new report out today by the Atlantic Council titled “How Do You Fix a Flying Computer? Seeking Resilience in Software-Intensive Mission Systems.” “Mission resilience must be a priority area of work for the defense community. Resilience offers a critical pathway to sustain the long-term utility of software-intensive mission systems, while avoiding organizational brittleness in technology use and resulting national security risks. The United States and its allies face an unprecedented defense landscape in the 2020s and beyond.” This resilience, is built upon three pillars, the authors write: robustness, which is the ability of a system to negate the impact of disruption; responsiveness, which is the ability of a system to provide feedback and incorporate changes on a disruption, and; adaptability, which is the ability to a system to change itself to continue operating despite a disruption. Systems, the report notes, are more than just the sum of its parts — hardware and software — but rather are much broader to include people, organizational processes and technologies. To date, DoD has struggled to manage complexity and develop robust and reliable mission systems, even in a relatively benign environment, the report bluntly asserts, citing problems with the F-35′s Autonomic Logistics Information System (ALIS) as one key example. “A conflict or more contested environment would only exacerbate these issues. The F-35 is not alone in a generation of combat systems so dependent on IT and software that failures in code are as critical as a malfunctioning munition or faulty engine — other examples include Navy ships and military satellites,” the authors write. “To ensure mission systems like the F-35 remain available, capable, and lethal in conflicts to come demands the United States and its allies prioritize the resilience of these systems. Not merely security against compromise, mission resilience is the ability of a mission system to prevent, respond to, and adapt to both anticipated and unanticipated disruptions, to optimize efficacy under uncertainty, and to maximize value over the long term. Adaptability is measured by the capacity to change — not only to modify lines of software code, but to overturn and replace the entire organization and the processes by which it performs the mission, if necessary. Any aspect that an organization cannot or will not change may turn out to be the weakest link, or at least a highly reliable target for an adversary.” The report offers four principles that defense organizations can undertake to me more resilient in future conflicts against sophisticated adversaries: Embrace failure: DoD must be more willing to take risks and embrace failure to stay ahead of the curve. Organizations can adopt concepts such as chaos engineering, experimenting on a system to build confidence in its ability to withstand turbulent conditions in production, and planning for loss of confidentiality in compromised systems. Improve speed: DoD must be faster at adapting and developing, which includes improving its antiquated acquisition policies and adopt agile methodologies of continuous integration and delivery. Of note, DoD has created a software acquisition pathway and is implementing agile methodologies of continuous integration and delivery, though on small scales. Always be learning: Defense organizations operate in a highly contested cyber environment, the report notes, and as the department grows more complex, how it learns and adapts to rapidly evolving threats grows in importance. Thus, it must embrace experimentation and continuous learning at all levels of systems as a tool to drive improvement. Manage trade-offs and complexity: DoD should improve mission system programs' understanding of the trade-offs between near-term functionality and long-term complexity to include their impact on systems' resilience. https://www.c4isrnet.com/cyber/2020/12/14/how-dod-can-improve-its-technology-resilience/

Toutes les nouvelles