The Quality Game
The pressure is on as companies strive to hit an ever-increasing number of moving targets. It’s not a pastime for the faint of heart.
Deep down, it may seem that quality is impossible to manage effectively because there are so many variables at play—after all, every process in your design or manufacturing operation impacts the final quality of the product, as well as the quality of the customer experience. Add in dealing with increasingly complex, global supply chains and the latest regulatory demands from the U.S. Food and Drug Administration (FDA)—often with limited resources—and you may feel like a scream or two is deserved when nobody is around.
In a new FDA report dated Oct. 31 and titled “Understanding Barriers to Medical Device Quality,” the agency states that serious adverse event reports related to medical device use have outpaced industry growth by 8 percent per year since 2001. Nearly 60 percent of these reports come from cardiovascular, in-vitro diagnostic (IVD) and general hospital/surgical devices. In addition, an analysis of root cause data reveals that failures in product design and manufacturing process control caused more than half of all product recalls.
Wherever you are in your operations, wherever you look, there is an abundance of opportunity to improve quality—design and reliability engineering (including validation of actual product use), post-production feedback regarding design and manufacturing, supply chain management (especially material and process change controls), quality metrics and measurement systems that go beyond minimum compliance, quality culture, risk management, performance management and post-market surveillance.
One of the biggest challenges to quality is the increasing complexity of medical devices and usage environments around the world.
According to the FDA, “many companies believe there issignificant pressure to enter a device market early to maximize payoffs due to intense competition; companies cite pressures to accelerate product launches instead of ensuring high quality
Other sources told Orthopedic Design & Technology that the primary focus for research and development is more for meeting timelines than embedding quality, and that cost pressures are forcing medical device manufacturers to lower their price points, which sometimes means taking risks by looking for low-costsuppliers that use lower-quality materials or have less robust controls in place.
“There are so many areas that need to be fully functional for device manufacturers to successfully navigate regulatory waters, at home and abroad, it will literally make you head spin,” said Christopher Devine, Ph.D., president of Devine Guidance International, a Las Vegas, Nev.-based quality and regulatory consulting firm specializing in supporting the medical device industry. “For example, in March 2010 a significant rescript of 93/42/EEC (the Medical Device Directive) took effect in Europe.
Almost overnight device manufacturers were forced to address clinical assessments, the reclassification of some devices, testing and labeling for phthalates and a dedicated essential-requirements checklist for devices categorized as personal protection equipment, just to name a few. Changes to the European directive for in vitro diagnostic devices are on the horizon. Aligning quality management systems with these types of regulatory changes, as well as proposed changes to the 510(k) submission and approval process in the U.S., take time and are expensiveto implement.”
Nelson Laboratories Inc. (NLI), a provider of full, life-cycle microbiology testing services in Salt Lake City, Utah, knows what quality issues are concerning the FDA from the increasing number of on-site audits it receives from its clients.
“Although these are typically general system audits, we can always tell what the FDA is focusing on based on what the auditors are looking for,” said Lillian Webb, regulatory specialist with NLI. “There are three general areas where we have seen an increase in diligence during the past year: supplier management, internal audits and root cause analysis/corrective and preventative action (CAPA).”
The FDA report states that “monitoring and management is widely identified as a continuing source of significant quality risk in the value chain. Risks are primarily around uncontrolled material or process changes, particularly when suppliers have an imperfect understanding of how their components affect end-product quality. It is especially challenging when dealing with a large, globalized supply base.”
NLI’s external audit schedule is growing fuller as the FDA pushes for more supplier qualification and management from its clients. These companies have told NLI that it no longer is sufficient to simply show the FDA they have added NLI as a qualified supplier and reviewed its ISO and FDA certificates.
Instead the FDA wants these companies to evaluate NLI services in detail, including identification of any inherent failure modes and installation of subsequent controls. The FDA also expects them to ensure that NLI thoroughly manages its own supply chain as well.
“This sometimes goes as far as clients calling our suppliers, requesting additional data for the products or services that have been performed for us, requesting changes from our vendors, etc.,” said Webb.
“As time-consuming as this can be, it shows the FDA that our clients are willing to audit us thoroughly and that NLI is willing to be as transparent as possible—always a good thing.”
Most device manufacturers struggle with one fundamental question: How often do suppliers require an audit? Sometimes small organizations never perform audits (always a major problem) and larger organizations often attempt to audit everyone.
“There really is some middle ground here,” said Devine, who recommends that device manufacturers place their supplier base into categories.
• Category 1: Sterilization or contract manufacturing suppliers (annual on-site audit mandatory)
• Category 2: Critical-component suppliers (on-site audit once every three years)
• Category 3: Component suppliers and other non-critical items (mail-in survey and ISO certificate)
• Category 4: Floor stock items (finger cots), etc. (mail-insurvey only)
“By classifying suppliers, device manufacturers can actually improve their quality while saving a significant amount of money relating to travel expenses associated with performing on-site audits,” said Devine. “If supplier problems arise, process audits tend to be more effective.”
ISO and the FDA place strong emphasis on continuous improvement and detection and correction of non-conformities. The best way to do this (especially prior to FDA audits) is to maintain a strong internal audit system. This also lets customers know the company is serious about identifying and mitigating risk, and investing in controls on a regular basis.
NLI has an internal initiative called “audit-ready” that essentially is a state of constant readiness for audits, whether they are internal or client audits, government audits or certification audits. Employees don’t need to be concerned when an unexpected sponsor or FDA examiner arrives unannounced to tour the facility because all systems are robust, in control and transparent to all levels of personnel. Webb indicated that every individual at NLI is ready and willing to present themselves and their duties to an auditor, at any time, without warning.
“You cannot be audit-ready if you are not willing to acceptconstructive criticism and make meaningful changes based on audit feedback,” said Webb. “Most companies are hesitant to expose their weaknesses. There may be some initial cost incurred if an auditor finds a major deficiency and a batch must be redone, a major process overhaul must be initiated or business is lost due to the severity of the error. But there would be a far greater cost incurred if the FDA found the deficiency, or a consumer sued or worse yet, died.”
Most of the feedback gained in audits, she pointed out, does not require complete system overhauls. Small incremental changes are the most common result of an audit and “they are the most valuable, as long as they are made with sustainability in mind,” she said.
Polymer Solutions Inc. (PSI) in Blacksburg, Va., a provider of analytical testing and consultative support to medical device companies, frequently conducts internal audits for all areas of its quality system and makes sure that every employee is trained to perform them.
“We also have employees perform internal audits in areas that fall outside of their departments in order for the audits to be more objective, which also allows them to experience new aspects of our organization,” said Caleb Rancourt, business intelligencespecialist for PSI.
Cynthia Rancourt, director of business operations, reported that she has seen an increase in audits of quality agreements at PSI.
“Quality agreements have generally not been scrutinized by auditors until now,” she said. “In a recent audit, a client requested to see the quality agreement and reviewed it item by item to make sure everything was done as stated, both by the client and by PSI as the supplier.”
Auditing the quality agreement is an effective way to revisit the expectations a client had at the beginning of the relationship. What is becoming more challenging is the level of specifications that clients are now adding to these agreements, beyond what is generally accepted as good manufacturing practice (GMP).
“Client legal departments, rather than the quality departments, seem to be adding specifics that go outside what is considered common GMP practice,” continued Cynthia. “An example is record retention. Seven to ten years is generally accepted as the standard for record retention for any documentation relating to client projects. PSI record retention procedures are written accordingly.
Recently a client quality agreement stated that records would be kept for 30 years. Some quality agreements are also starting to specify additional language on the report templates and certificates of analysis. Keeping track of exceptions to standard procedures is difficult to do without having to create yet another system involving more documentation and checklists, which places additional administrative demands on scientists who have been hired to produce reliable laboratory data with short deadlines—driving up costs.”
The FDA indicates that quality tools and processes in the medical device value chain have not kept pace with the increase in device complexity. Whole product design, defining critical-to-quality metrics and post-production monitoring were commonly cited as areas of quality risk. In addition, few companies use formal statistical tools in a disciplined way to accurately capture and analyze critical requirements.
“Risk assessment tools like design and process failure mode and effects analysis (FMEA) are often not developed, applied appropriately or updated frequently enough to incorporate substantial post-production feedback from the field,” according to the FDA.
Adverse events must be followed by a root cause analysis to determine the cause of the error, as well as a corrective and preventative action to fix the problem—two important steps that many medical device companies fail to do properly.
“Root cause is definitely an area of weakness for many companies, mainly due to a lack of training, time and resources to actually fix the root cause of any issue, which tend to be deeper, more intensive problems,” said Webb. “Band-aids are cheaper.”
Human error often is a hasty conclusion that is usually based on insufficient evidence, with the CAPA response being retraining—something that is relatively easy to do and low cost, but doesn’t fix the problem. A thorough root cause analysis (RCA) investigation often reveals a much deeper issue is to blame.
“If only the top levels of an event (human error or equipment failure) are investigated, only one problem will be solved,” according to the FDA’s report. “Unfortunately, some RCA methods fail to dig far enough. Often, organizations will get down to the management-systems level and stop at that point, primarily because corrective actions and recommendations must be actionable and measurable, which may be more difficult to do at the organizational cultural level.”
Webb noted the FDA lately has been focusing on the adequacy of root cause conclusions, the effectiveness of subsequent actions taken and the adequacy of a company’s verification of that effectiveness.
“Our external auditors expect us to not only have a system in place to document non-conformances, but to take meaningful action to determine the true root cause, investigate the impact and correct and prevent future occurrences,” she explained.
Webb said that NLI continues to improve its CAPA and quality event (non-conformance) systems by incorporating feedback from internal and external audits. Recently the company added the FDA’s 4x4 risk score (frequency x severity) to the non-conformities form, which helps determine risk levels for various systems and appropriate actions for mitigating those risks. A very detailed root cause checklist also was developed to aid in investigations.
“We use a matrix that we borrowed from Pathwise (www.pathwise.com/library/Risk_IVD_Kits.pdf), but updated with the FDA’s definitions of severity,” said Webb. “I think most companies are still analyzing risk in a full FMEA or HACCP (hazard analysis and critical control points) model, which is great if you have the time, but not reasonable when you are analyzing the risk of every non-conformance you have.”
Webb indicated NLI is investing considerable resources in its internal audit program and so far “results have been fantastic.”
That means no errors or corrections, right? Wrong.
“For us fantastic results actually means finding significant issues where we can invest resources and improve our systems,” said Webb. “It might sound strange, but we aren’t particularly satisfied when our internal audits yield small corrections needed, rather than areas where real improvement can be made. No matter how robust and refined your system is, good change can always be made.”
As quality management becomes more complex, more medical device manufacturers are turning to sophisticated, real-time software management systems that control/monitor nearly every conceivable process or metric—a short list includesdefects, customer complaints, safety issues, supplier quality,
laboratory management, document control and content management [especially popular as the FDA demands more data for the 510(k) process] audits and findings, R&D problem reports, nonconformance management, adverse events and correctiveactions, employee certification and training, equipment maintenance and calibration, internal and external audits, and supply chain management.
Costs vary, of course, depending on the scale of the operation and its specific needs. On average, for a medium-sized medical device manufacturer, to purchase and implement a high-quality, enterprise compliance-quality-risk management package with implementation ranges from about $125,000 to $1 million or more.
Return on this investment is anywhere from six months to several years—the combined positive effect of a variety of improvements and benefits including higher productivity, real-time detection and control of adverse events, better quality (fewer defects), faster production times, reduced operational costs, streamlined FDA reporting and improved customer satisfaction (an intangible that may pack the biggest value of all if it translates into expanded or extended contracts).
These software solutions also are available as software as a service (SaaS) that can deployed rapidly to enterprise systems via the Web, eliminating the need for installation, implementation, training and management for an in-house system. SaaS can be delivered on a subscription basis from a secure, 24/7 hosting facility and is completely scalable, allowing for easy expansion or additional application installation. It especially is useful for sharing information and enhancing real-time collaboration, making it ideal for global companies with multiple sites.
Quality must start with the president and CEO if it is going to thrive in an organization. The FDA reported that “interviewees whose companies experienced quality issues typically did not track quality metrics at the executive level, and quality was considered the domain of a specific organization rather than a CEO-level agenda item. Regular management visibility into key quality (not merely compliance) metrics was considered vital to embedding a quality centric culture.”
“Significant improvement cannot be sustained without the buy-in of the top management,” she said.
“Many companies suffer when decision-makers do not take the risks and expend the resources that go hand-in-hand with continuous improvement.”
According to discussions with hundreds of her auditors, Webb estimates that about 70 percent of large medical device manufacturers lack true buy-in at the top.
“They are given budgets to simply fix issues as they occur, not to dig deep and invest in a preventative solution,” she said. “For start-ups, where there is still a fair amount of venture capital and a small top-management group with a more idealistic attitude, that number is closer to half.”
Quality and regulatory personnel must actually read the regulations for the markets for which their devices are going to be placed.
“You would be surprised at the number of folks in the industry who lack actual training regarding the regulations,” said Devine. “Typically, regulatory folks are pretty much in tune with regulations; not so much for some of the quality folks and engineers. The lack of knowledge becomes readily apparent with the engineers and manufacturing personnel. I am getting more requests from clients who want me come to their sites and spend the day explaining regulatory requirements to their staffs.”
Quality and regulatory professionals must be able to effectively execute their job responsibilities. Maintaining and improving quality is a 24/7 job.
“Fail to understand the dynamic nature of quality by becoming complacent and your competition will quickly capitalize on your organization’s complacency,” warned Devine.
What is the hardest part of quality management for mostorthopedic companies?
“Sustainability,” said Devine. “For example, once a quality management system (QMS) has been implemented, it takes adequate resources, training, and managerial oversight to keep the QMS fully functional and compliant. Fail to adequately staff or manage the QMS and the system will fail. Once the deviations start pouring in from the notified bodies or Form 483s from the FDA, all bets are off.”
There is such a thing as ideal quality, where unlimited resources are spent achieving the pinnacle of quality perfection. Unfortunately, very few (if any) companies have the resources available to practice perfect quality—even if they did, there is a level reached where the contribution is not value-added.
“This level is attained when the risk of inadequate quality is not high enough to justify the attention paid to it,” said Webb. “For any company, large or small, quality actions should be based upon risk. This risk should be determined through documented, justifiable means, and should always be tied to both risk to the consumer and/or client, and risk to the business function. By identifying where your risks are most crucial, you can allocate the necessary resources to mitigate that risk.”
Mark Crawford is a full-time freelance business and marketing/communications writer based in Madison, Wis. His clients range from startups to global manufacturing leaders such as Kohler. He also writes a variety of feature articles for regional and national publications and is the author of five books. Contact him at firstname.lastname@example.org.