If there is one lesson to draw from this year's Boeing 737 Max debacle, it is that software glitches can have catastrophic consequences.
That isn’t exactly a new revelation. There are multiple examples in recent decades of software bugs leading to loss of life and other forms of destruction. The story of the Therac-25 computerized radiation therapy machine is one of the most evocative. Produced in the 1980s, the Therac-25 has become a classic example of the potential for software failures to cause injuries and deaths, according to Anura Fernando of UL. Between 1985 and 1987, at least six patients received massive overdoses of radiation. The manufacturer, Atomic Energy of Canada Ltd., designed the system “to take advantage of computer control from the outset,” as Nancy Leveson and Clark Turner wrote in the journal “Computer” in 1993. A single person programmed the machine, modifying code from older devices and apparently documenting little of the process.
The Therac-25 case has been “well-studied academically,” said Fernando, who is UL’s chief innovation architect, medical systems interoperability and security. “A lot of the early medical device software quality requirements were formulated around that case study.”
The fallout from the software failure paved the way for modern software quality requirements. It also helped drive the recognition “that our society was becoming more and more dependent on software,” added Fernando.
Health care industry’s software maturity has improved considerably in the intervening decades. But software problems continue to be one of the most common causes of medical device recalls.
Awareness of the threat of cybersecurity in health care is also on the rise. There is often a gulf, however, between the risk of cyber vulnerabilities and the underlying software faults that have been a concern for decades. “Cybersecurity is kind of this new topic [in health care],” Fernando said. “It’s kind of scary and an unfamiliar topic for a lot of people. But the basic concern has been there for a long time.” It’s just that now, the worry is that adversaries could step up efforts to exploit the software bugs that have been present for as long as software has existed.
The trend to imbue medical devices with so-called Internet of Things functionality substantially increases the awareness of the risk.
“If you look back historically to the late 1990s when the FDA first issued software-related guidance, nobody was really talking about cybersecurity,” Fernando said. At that time, however, there was a strong focus on quality, he explained. “People knew that software was an intangible subsystem within medical device hardware. And if that software was compromised, it could cause really bad things to happen.”
About a decade ago, the FDA began to increase its focus on cybersecurity in its communications with industry. In 2014, the agency released the first pre- and post-market cybersecurity guidance documents. “The things that are called out in [in those documents] — many of the cybersecurity software issues — are fundamentally software quality issues. We’ve known about many of them for decades,” Fernando said. One example of such a problem is buffer overflow. “Even non-programmers have often heard that term. It’s a software phenomenon that can be exploited to allow you to access unintended, and potentially protected, memory spaces,” he added. As the Wikipedia article on the subject observes, a “technically inclined user” can use a buffer allow to cause a variety of other problems such as overwriting local variables or overwriting the return address in a stack frame.
The buffer overflow was likely one of the earliest vulnerabilities hackers exploited, but now there are thousands of vulnerabilities. “And the processes that can be used to address those [vulnerabilities] are fundamentally software quality processes,” Fernando said.
Such quality processes are detailed in standards such as ISO 13485, which focuses on medical device quality management systems. UL’s 2900 series of standards covers similar territory — for networked products broadly, including medical devices. FDA has recognized UL 2900-2-1 to help assess the cybersecurity risk of networked medical devices.
Such standards drive manufacturers to look at their development processes to determine where systematic defects can be minimized. “That’s really important for software, both from a quality and safety and effectiveness perspective, but now from a cybersecurity point of view, as well,” Fernando said. “As you dissect the cybersecurity requirements you see emerging in multiple sectors, they are really predicated on fundamental software quality assurance.”
When cybersecurity began to emerge as a priority in the medical device industry, the threat initially seemed partly hypothetical. Cybersecurity researchers regularly demonstrated how to compromise devices ranging from pacemakers to insulin pumps in the early 2010s, but it was difficult to gauge how substantial the risk was.
In the interim, cyberattacks on health care institutions have become more common. Ransomware, in particular, is problematic. “Without a question, it’s at the top of the list,” Fernando said.
The adversaries who launch a ransomware campaign against a medical establishment have a variety of motivations. On the one hand, it is often an effective way for a hacker to make money. On the other, ransomware attacks provide a convenient way for an attacker to wreak havoc while obscuring the true intent.
Speaking in general on the subject at Defcon, Bryson Bort, the founder of Scythe, said: “Ransomware has two benefits: One, it is destructive. Two, it can be destructive by accident.” That is, it obscures the attackers’ intent.
Fernando had a similar assessment. “There’s this obvious motivation of trying to get a financial return by holding the infrastructure ransom,” he said. “Then there’s the motivation of nation-state-sponsored hackers probing health care infrastructure as critical infrastructure to be debilitated under the guise of ransomware.” That is, a ransomware attack can help a threat actor understand health care infrastructure and discover its vulnerabilities. For a nation-state, launching such an attack is dramatically less expensive than a traditional military operation.
The silver lining of such somber developments is the fact that awareness of the problem is growing, which is leading to an uptick in hiring cybersecurity professionals in the medical field. “If you just look at the employment websites, you see a lot of job postings in the health care sector for cybersecurity leaders both on the medical device manufacturing side, as well as the health care delivery side,” Fernando said.
There is, however, a shortage of experienced cybersecurity workers that plagues industries at large. One of the big-ticket items from the Health Care Industry Cybersecurity Task Force report was to address “the workforce shortage now that we’re currently seeing with cybersecurity,” said Fernando, a coauthor of the document. “There’s also the need to secure the supply chain,” he added. “There’s improving capability and maturity of manufacturers in a scalable way. There’s helping increase [cybersecurity] awareness within hospitals.”
The health care industry can be generally slow to adapt, owing partly to its complexity and the number of constituents involved. To help drive further progress, the Healthcare and Public Health Sector Coordinating Council is working to help drive measurable improvement in terms of the recommendations from the Health Care Industry Cybersecurity Task Force report. The council was recently “reactivated and reengaged” to work on ways to execute on those recommendations, according to Fernando. Ultimately, the work of such organizations has paved the way for a joint security plan that provides “public domain processes and information tools that can help medical device manufacturers and hospitals better collaborate on dealing with this community issue of security,” Fernando said.
Ultimately, the work of such organizations could play an essential role in continuing to improve software quality as well.
“Quality is never an accident,” as the 19th century English writer and artist John Ruskin surmised. “It is always the result of high intention, sincere effort, intelligent direction and skillful execution; it represents the wise choice of many alternatives.”