Linda Braddon, Ph.D., President and CEO, Secure BioMed Evaluations05.19.17
The European Union’s requirements for CE marking significantly changed with last summer’s fourth official revision of MEDDEV 2.7/1. From the standpoint of those who ensure a project’s documents are compliant and properly submitted for approval, the impact of these changes is substantial. Though there are many parts of a technical file or design dossier needed to obtain CE marking, this article will focus specifically on the Clinical Evaluation Report (CER).
CERs are necessary in order to sell orthopedic devices in Europe. Guidance specifying its requirements are detailed in MEDDEV 2.7/1 (Revision 4), published in June 2016. In basic terms, the clinical evaluation report is a critical analysis of available clinical data pertinent to a device that exposes both the risks and the benefits of that device. In the past, for many “me-too” orthopedic products, a detailed literature review was satisfactory to argue that a device was equivalent in safety and performance to similar products. A successful CER uses clinical evidence to show that a device meets its intended purpose without exposing patients (or users) to new or additional risks. If the product poses a new risk, the CER explains how the benefits of the therapy will outweigh that risk. In past years, companies performing a robust literature review and building a reasonable case for product equivalency fulfilled their CER mandate without the help of clinical data or a post-market clinical follow-up. Under the latest MEDDEV 2.7/1 revision, however, companies must submit additional clinical proof with their CERs.
Defining the CER
According to MEDDEV 2.7/1 Revision 4, a clinical evaluation is defined as “a methodologically sound ongoing procedure to collect, appraise and analyze clinical data pertaining to a medical device and to evaluate whether there is sufficient clinical evidence to confirm compliance with relevant essential requirements for safety and performance when using the device according to the manufacturer’s Instructions for Use.” A detailed evaluation of the CER and supporting documentation is mandatory for CE mark approval. Additionally, CERs must be updated on a regular basis, with the frequency dependent on such factors as product risk, level of innovation, and trending adverse events. Periodic reviews and CER updates must also consider post-market data, risk management activities for the device, and other metrics related to the product. For example, a simple fixation screw—sans an industry-wide recall or patient complaint—would require less frequent updates than a novel knee replacement device.
The foundation of a CER is built with a protocol that determines the methodology for keyword searches, the exclusion criteria for narrowing down the number of articles for review, and the approach for evaluating and summarizing data sources. The search strategy is one of the most critical aspects of the CER. It should identify both favorable and unfavorable results, and be repeatable. Once articles are identified, pre-defined exclusion criteria can be used to whittle the list of articles for review. Bear in mind there are varying calibers of published literature and the results do not necessarily carry the same weight. A single patient case study, for example, is not nearly as reliable or compelling as a 300-patient randomized clinical trial. It usually is appropriate to exclude articles with small numbers of patients because the statistical significance cannot be justified. Likewise, abstracts can be used to evaluate the strength of an article but may not be suitable to determine whether the item should be included, and they should certainly never be sourced in summarizing an included article. Since full text articles of the incorporated data are typically included as an appendix to the CER, retrieval costs for all articles should be part of a CER budget. There is no prescriptive methodology for documenting article inclusion, but I’ve found that a simple excel spreadsheet works well, with rows identifying the chosen items and the rationale for their selection as well as a scoring system to define the “weight” of an article. Be cognizant that the search and final compilation of included data sources should be reproducible by another reviewer using the same protocol. Clear documentation of the decision-making process will expedite both the regulatory review and future update efforts.
After identifying the included articles, the evaluation process begins. It is necessary to identify and evaluate the quality of data, determine its relevance to the proposed device, evaluate the potential for redundancy with other included factual sources, and weigh the data’s contribution to the clinical evaluation. Since the CER should contain a protocol with pre-defined appraisal criteria, the article evaluations may be tedious and time consuming. But they should not be too complicated (remember, an independent reviewer should easily be able to duplicate the results). Make the appraisals clear and consistent. The evaluation should provide sufficient detail for understanding the search criteria, available data, all assumptions made, and conclusions reached. In addition, the CER’s contents must be cross-referenced to the relevant documents that support them since the CER is considered a stand alone document. It is more effective to state facts supported by objective evidence than draw conclusions that could be considered as opinion. The more innovative or novel the device, the more information is needed on the development of the technology.
Although the literature evaluation section is critical, there are other parts of a CER that are equally important. Specifically, these components include a device description, scope of the evaluation, risk management documents, state-of-the-art analysis, discussion of equivalency (or differences), and planned post-market surveillance activities. In my experience, the areas that receive the most negative feedback from notified bodies are the state-of-the-art explanation coupled with the risk/benefit analysis, justification for equivalency, and the post-market surveillance section.
State-of-the-Art and Risk/Benefit Analysis
The state-of-the-art section is an exhaustive list of alternative therapy options for patients. In orthopedics, this list often includes therapy options such as behavior modifications, physical therapy, drugs, and surgery. Overview articles that may or may not be included in the literature review can be very important in the state-of-the-art section. This segment should be an exhaustive list of available options with a clear description of risk profiles, and it should demonstrate that patient benefits are well understood. This section is critical to the risk/benefit review. Currently available technologies for a certain indication are typically well-defined with a range of risks and benefits associated with the modality. In order to bypass clinical data, the risk/benefit profile must fall within the predetermined limits defined by the existing therapies. A CER that reveals residual risks, uncertainties, or unanswered questions will most certainly necessitate a post-market clinical follow-up.
Equivalency
Those who have written 510(k) applications for the U.S. Food and Drug Administration (FDA) understand the significance of a substantial equivalence table in the decision-making process. Demonstrating equivalence is critical in the CER as well, and is typically reviewed in three distinct areas: clinical, technical, and biological. Clinical equivalency encompasses use for the same clinical condition for the same intended purpose, used at the same body site on similar patient populations, all while delivering a similar performance.
Technical equivalency is important, too. The subject device should be designed similarly to other marketed devices, with comparable specifications and technical features. The principles of operation and critical performance requirements should also align to existing products, as should biological compatibility and patient contacting materials. Assessment of design details to the “predicate” device is critical in demonstrating equivalence. One way to highlight similarities between products is through a substantial equivalence table that uses a single device for comparison and notes all differences. A notified body—much like the FDA—will reject CERs that try to prove equivalence using individual characteristics from multiple devices. Obvious differences in technology will not go unnoticed. Be critical in assessing the technology. If aspects of multiple devices must be used to make an argument for equivalence or if a technological change raises a potential new risk, the CER will likely need clinical data to pass muster with European regulators.
Post-Market Surveillance
Post-market surveillance information is very important in showing routine monitoring of a medical device’s clinical performance. Incoming data from monitoring activities should be evaluated against the risk management file to ensure new risks are identified. New risks acknowledged through clinical experience may lead to labeling updates. The current trend in post-market surveillance is the “pro-active” surveillance requirement. In the past, notified bodies would readily accept a robust complaint monitoring, but now a post-market registry is the minimum requirement (in addition to existing passive post-market scrutiny). A more common trend, however, is the Post Market Clinical Follow-Up plan, which looks more like a clinical trial than simple complaint monitoring.
The Case for Clinical Data
A CER based solely on literature can only be effective if there is sufficient published data on a similar product, so conclusions can be drawn on the subject device’s anticipated safety and performance. I have found that a literature-based CER is inadequate with novel device designs because it is difficult to prove equivalency based on a single product. As previously noted, using design features from multiple devices to evaluate the new technology will likely be frowned upon by notified bodies and consequently require the addition of supportive clinical data. Areas that often necessitate the need for clinical data are new design features, new materials, new intended purpose, new claims, and new or undefined risks. One of the biggest mistakes companies make is trying to convince a notified body that a novel technology inherently has no new risks and not using clinical experience to answer critical concerns. Turning a blind eye to clinical data will not change the answer for notified bodies, but it most certainly will cost companies precious time and resources. Don’t make that mistake. Embrace clinical data and use it to create a more compelling CER.
Linda Braddon, Ph.D., is president and CEO of Secure BioMed Evaluations. She works with emerging and established companies to prove regulatory, quality, and technical support to both the medical device and biologics industries. Dr. Braddon has a B.S. degree in engineering from Mercer University along with a master of science degree and a Ph.D. in mechanical engineering with a specialization in bioengineering. Her 20 years of experience includes extensive work in orthopedics, dental implants, ophthalmology, respiratory, urology, hydrogels, dura mater substitutes, wound coverings, orthotic devices, and antimicrobial agents.
CERs are necessary in order to sell orthopedic devices in Europe. Guidance specifying its requirements are detailed in MEDDEV 2.7/1 (Revision 4), published in June 2016. In basic terms, the clinical evaluation report is a critical analysis of available clinical data pertinent to a device that exposes both the risks and the benefits of that device. In the past, for many “me-too” orthopedic products, a detailed literature review was satisfactory to argue that a device was equivalent in safety and performance to similar products. A successful CER uses clinical evidence to show that a device meets its intended purpose without exposing patients (or users) to new or additional risks. If the product poses a new risk, the CER explains how the benefits of the therapy will outweigh that risk. In past years, companies performing a robust literature review and building a reasonable case for product equivalency fulfilled their CER mandate without the help of clinical data or a post-market clinical follow-up. Under the latest MEDDEV 2.7/1 revision, however, companies must submit additional clinical proof with their CERs.
Defining the CER
According to MEDDEV 2.7/1 Revision 4, a clinical evaluation is defined as “a methodologically sound ongoing procedure to collect, appraise and analyze clinical data pertaining to a medical device and to evaluate whether there is sufficient clinical evidence to confirm compliance with relevant essential requirements for safety and performance when using the device according to the manufacturer’s Instructions for Use.” A detailed evaluation of the CER and supporting documentation is mandatory for CE mark approval. Additionally, CERs must be updated on a regular basis, with the frequency dependent on such factors as product risk, level of innovation, and trending adverse events. Periodic reviews and CER updates must also consider post-market data, risk management activities for the device, and other metrics related to the product. For example, a simple fixation screw—sans an industry-wide recall or patient complaint—would require less frequent updates than a novel knee replacement device.
The foundation of a CER is built with a protocol that determines the methodology for keyword searches, the exclusion criteria for narrowing down the number of articles for review, and the approach for evaluating and summarizing data sources. The search strategy is one of the most critical aspects of the CER. It should identify both favorable and unfavorable results, and be repeatable. Once articles are identified, pre-defined exclusion criteria can be used to whittle the list of articles for review. Bear in mind there are varying calibers of published literature and the results do not necessarily carry the same weight. A single patient case study, for example, is not nearly as reliable or compelling as a 300-patient randomized clinical trial. It usually is appropriate to exclude articles with small numbers of patients because the statistical significance cannot be justified. Likewise, abstracts can be used to evaluate the strength of an article but may not be suitable to determine whether the item should be included, and they should certainly never be sourced in summarizing an included article. Since full text articles of the incorporated data are typically included as an appendix to the CER, retrieval costs for all articles should be part of a CER budget. There is no prescriptive methodology for documenting article inclusion, but I’ve found that a simple excel spreadsheet works well, with rows identifying the chosen items and the rationale for their selection as well as a scoring system to define the “weight” of an article. Be cognizant that the search and final compilation of included data sources should be reproducible by another reviewer using the same protocol. Clear documentation of the decision-making process will expedite both the regulatory review and future update efforts.
After identifying the included articles, the evaluation process begins. It is necessary to identify and evaluate the quality of data, determine its relevance to the proposed device, evaluate the potential for redundancy with other included factual sources, and weigh the data’s contribution to the clinical evaluation. Since the CER should contain a protocol with pre-defined appraisal criteria, the article evaluations may be tedious and time consuming. But they should not be too complicated (remember, an independent reviewer should easily be able to duplicate the results). Make the appraisals clear and consistent. The evaluation should provide sufficient detail for understanding the search criteria, available data, all assumptions made, and conclusions reached. In addition, the CER’s contents must be cross-referenced to the relevant documents that support them since the CER is considered a stand alone document. It is more effective to state facts supported by objective evidence than draw conclusions that could be considered as opinion. The more innovative or novel the device, the more information is needed on the development of the technology.
Although the literature evaluation section is critical, there are other parts of a CER that are equally important. Specifically, these components include a device description, scope of the evaluation, risk management documents, state-of-the-art analysis, discussion of equivalency (or differences), and planned post-market surveillance activities. In my experience, the areas that receive the most negative feedback from notified bodies are the state-of-the-art explanation coupled with the risk/benefit analysis, justification for equivalency, and the post-market surveillance section.
State-of-the-Art and Risk/Benefit Analysis
The state-of-the-art section is an exhaustive list of alternative therapy options for patients. In orthopedics, this list often includes therapy options such as behavior modifications, physical therapy, drugs, and surgery. Overview articles that may or may not be included in the literature review can be very important in the state-of-the-art section. This segment should be an exhaustive list of available options with a clear description of risk profiles, and it should demonstrate that patient benefits are well understood. This section is critical to the risk/benefit review. Currently available technologies for a certain indication are typically well-defined with a range of risks and benefits associated with the modality. In order to bypass clinical data, the risk/benefit profile must fall within the predetermined limits defined by the existing therapies. A CER that reveals residual risks, uncertainties, or unanswered questions will most certainly necessitate a post-market clinical follow-up.
Equivalency
Those who have written 510(k) applications for the U.S. Food and Drug Administration (FDA) understand the significance of a substantial equivalence table in the decision-making process. Demonstrating equivalence is critical in the CER as well, and is typically reviewed in three distinct areas: clinical, technical, and biological. Clinical equivalency encompasses use for the same clinical condition for the same intended purpose, used at the same body site on similar patient populations, all while delivering a similar performance.
Technical equivalency is important, too. The subject device should be designed similarly to other marketed devices, with comparable specifications and technical features. The principles of operation and critical performance requirements should also align to existing products, as should biological compatibility and patient contacting materials. Assessment of design details to the “predicate” device is critical in demonstrating equivalence. One way to highlight similarities between products is through a substantial equivalence table that uses a single device for comparison and notes all differences. A notified body—much like the FDA—will reject CERs that try to prove equivalence using individual characteristics from multiple devices. Obvious differences in technology will not go unnoticed. Be critical in assessing the technology. If aspects of multiple devices must be used to make an argument for equivalence or if a technological change raises a potential new risk, the CER will likely need clinical data to pass muster with European regulators.
Post-Market Surveillance
Post-market surveillance information is very important in showing routine monitoring of a medical device’s clinical performance. Incoming data from monitoring activities should be evaluated against the risk management file to ensure new risks are identified. New risks acknowledged through clinical experience may lead to labeling updates. The current trend in post-market surveillance is the “pro-active” surveillance requirement. In the past, notified bodies would readily accept a robust complaint monitoring, but now a post-market registry is the minimum requirement (in addition to existing passive post-market scrutiny). A more common trend, however, is the Post Market Clinical Follow-Up plan, which looks more like a clinical trial than simple complaint monitoring.
The Case for Clinical Data
A CER based solely on literature can only be effective if there is sufficient published data on a similar product, so conclusions can be drawn on the subject device’s anticipated safety and performance. I have found that a literature-based CER is inadequate with novel device designs because it is difficult to prove equivalency based on a single product. As previously noted, using design features from multiple devices to evaluate the new technology will likely be frowned upon by notified bodies and consequently require the addition of supportive clinical data. Areas that often necessitate the need for clinical data are new design features, new materials, new intended purpose, new claims, and new or undefined risks. One of the biggest mistakes companies make is trying to convince a notified body that a novel technology inherently has no new risks and not using clinical experience to answer critical concerns. Turning a blind eye to clinical data will not change the answer for notified bodies, but it most certainly will cost companies precious time and resources. Don’t make that mistake. Embrace clinical data and use it to create a more compelling CER.
Linda Braddon, Ph.D., is president and CEO of Secure BioMed Evaluations. She works with emerging and established companies to prove regulatory, quality, and technical support to both the medical device and biologics industries. Dr. Braddon has a B.S. degree in engineering from Mercer University along with a master of science degree and a Ph.D. in mechanical engineering with a specialization in bioengineering. Her 20 years of experience includes extensive work in orthopedics, dental implants, ophthalmology, respiratory, urology, hydrogels, dura mater substitutes, wound coverings, orthotic devices, and antimicrobial agents.