FDA (CBER) Draft Guidance for Industry: BECS (Blood Establishment Computer Systems) Validation in the User’s Facility

FDA (CBER) Draft Guidance for Industry: BECS (Blood Establishment Computer Systems) Validation in the User’s Facility

  

blood via clipart

 

Overview

 

The following post comes from the October 2007 Draft Guidance as seen in the following link: http://www.fda.gov/BiologicsBloodVaccines/GuidanceComplianceRegulatoryInformation/Guidances/Blood/ucm072560.htm .  This guidance addresses high level validation requirements for computer systems utilized within the Blood Industry.  Specifics of the validation methods can be found at ‘General Principles of Software Validation; Final Guidance for Industry and FDA Staff (January 11, 2002)’, http://www.fda.gov/cdrh/comp/guidance/938.html .  The BECS (Blood Establishment Computer Systems) 510(k) premarket notification will not be addressed within this post but details can be found in the following link: ‘Guidance for Industry and FDA Staff: Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices (May 11, 2005)’, http://www.fda.gov/cdrh/ode/guidance/337.html . This post is relevant to Blood Establishments who plan to create or utilize BECS within their regulated operations.

 

 

BECS Defined

 

BECS (Blood Establishment Computer Systems) is the classification of computer hardware, software, peripheral devices, end users, and related documentation and standard operating procedures.  The FDA considers BECS a medical device and subject to 510(k).  Additional requirements to consider are equipment classifications under Title 21 Code of Federal Regulations (CFR) 606.60, and automated or electronic equipment classifications under 21 CFR 211.68.

 

Not all computer systems fall under the BECS classification even if they are utilized within a Blood Establishment.  Obviously a system utilized for HR or PTO will not fall under this classification. BECS classification is specified for systems designed to be utilized in a blood establishment that is intended for use in the manufacturing / collection of the blood or blood related product, diagnosis of donors, or in the release blood and blood components. Some of the intended uses of BECS include:

• use during the manufacturing process for determining donor suitability/eligibility and release of the blood or blood component as suitable for transfusion or further manufacture; and
• use in transfusion services to perform compatibility testing and other related functions.

 

 

 

Validation Components

 

As far as draft guidance documents are concerned, this is one of the most clear and concise guides and it does a good job in conveying the overall process.  Below is the list of  topics that were addressed.  Each section will be briefly summarized.

 

• Vendor Selection
• Validation Coverage
• Risk Assessment
• Validation Procedures
• Validation Plan
• Validation Activities
• Validation Report
• Validation after a Change
• System Documentation

 

 

Vendor Selection

 

The recommendation is that the BECS be selected based upon the needs of the Blood Establishment.  This can be accomplished by establishing a User Requirements document during the early stage of development or system selection.  In addition, a Vendor Audit should be performed prior to a selection of COTS (Configurable Off The Shelf).  This guidance also recommends monitoring all recalls and reports of adverse events related to your BECS.  Another resource that should be utilized is communication with the vendor related to any major bugs or other hardware / software defects.

 

Validation Coverage

 

Validation of your Blood Establishment Computer System should reflect and anticipate the system’s actual use in your establishment. You should validate your system using your own personnel and your own test plans written for your specific intended use of the software. You may wish to retain a consultant for direction and assistance for this project.

 

The validation process should start with clearly defined usage as mentioned in the previous section.  This usage should clearly reflect actual configuration and utilization.  Requirements should provide a direction for testing and the validation process.  A validation consultant can streamline this process for establishments who lack the staffing or experience.

 

We also recognize that it is a common practice for the software manufacturer to provide test cases to blood establishments for use in system validation. We recommend that you carefully consider your own intended use of the software and your internal policies and procedures and add or change test plans as appropriate to ensure that the software will accurately and repeatedly meet your requirements.

 

Not all computer system / software vendors have adequate validation protocols.  As stated above, it is up to the end user to confirm that the protocols meet the intended use of the system.  It is often the case that vendors will do an adequate job defining the functional testing but fail at usage testing.  Since most software is configurable, the usage testing will differ for almost every install.  In some cases, the vendor will lack the appropriate level of quality within the organization to create any compliant, validation, testing documentation.  It is imperative to perform a thorough review of all protocols before accepting a vendor provided solution.

 

Risk Assessment

 

The level of confidence and, therefore, the level of validation effort needed, vary depending upon the potential hazards posed by the automated functions of the system. Your test plan and test cases should be developed based on a risk assessment. We, therefore, recommend that you perform a risk assessment early in the validation process. After performing that assessment, you should determine the degree of validation necessary based on the identified risks, and then develop your test plan and test cases accordingly. For example, because of the potentially fatal consequences of a transfusion based on an improper crossmatch between donor and recipient, the validation of the electronic or computer crossmatch would require a higher degree of testing.

 

I’ve posted a Risk Assessment for the Blood Industry which should be reviewed.  Risk based approach to validation is the common approach throughout all industries.  The basic approach is that if it can happen, it will happen.  However, the focus should be on establishing a high level of confidence for the system, not identifying all remote possibilities.

 

Validation Procedures

You must develop SOPs for your validation activities as indicated by 21 CFR 211.68(a), 211.100(a), and 606.100(b)(15). We recommend that your validation SOPs include, but not necessarily be limited to, the following:

• writing a validation plan
• performing a risk assessment
• writing a validation report
• addressing change control
• writing a test case
• amending a test case
• handling validation deviations
• validating after a change
• performing system maintenance

This section require no further comment expect that I would stay away from the term ‘deviation’ and utilize ‘exceptions’ instead.  Under no circumstances should ‘deviations’ within validation be entered into the production system.  Again, see my previous posts for further discussion.

 

Validation Plan

You should define and control how you will validate BECS through the use of a plan. The validation plan defines “what” the validation effort should accomplish. Validation plans specify areas such as scope, approach, resources, schedules, the types and extent of activities, tasks, and work items, and the approval process.

 

Having performed many validations for a variety of companies, blood centers, and institutions it should be mentioned that often the validation plans are not completed and approved prior to testing.   AABB specifically states that validation plans should be created before the start of any testing.  Obviously, it would not be compliant to write the release requirements after the testing was performed.

 

Validation Activities

Consider the following points as you prepare to perform validation activities:

We recommend that you perform validation in the “test environment” or “test partition” of your system. After successful validation, you or  your software manufacturer should copy or move the file configuration on which you performed your validation into the production  environment.

Since software can be copied typically without error, it is not necessary to copy the environment from the ‘test’ to the production environment.  Doing so may leave unwanted testing artifacts.  It is better to create a test site that mirrors the configuration of the production environment and utilize a clean production site when testing is completed.

We recommend that you follow a pre-defined written test plan. Each test case in the plan should include the input, expected output, actual output, acceptance criteria, whether the test passed or failed, the name or initials of the person performing the test, and the date the test was performed. Test cases should include normal results (results within the “normal range”), abnormal results (unacceptable results or those outside the “normal” range) and boundary results or values. Boundary testing is testing at the boundary of a specification, in other words, at the limit, just below the limit, and just over the limit. Boundary values are “off-by-one errors.” An example of boundary testing is testing for a hematocrit of 37%, 38% and 39% if the cutoff (or boundary value) for hematocrit is 38%. You should also test for “absurd” results (invalid or nonsense results). Examples of absurd or unexpected values might be an alpha result in a numeric field, a hematocrit of 110% or a blank (or leading blank) in a result field.

Obviously, the abnormal results and boundary ranges only apply to an analytical testing system which returns a specific value, set of values, or range.  For output that does not fall under this classification, expected results is sufficient. 

You must retain validation records (21 CFR 211.68(b), 211.100(b), 606.160(b)(5)(ii)). Your records should include documented evidence of all test cases, test input data and test results. Test results should include screen prints. For traceability purposes and to facilitate quality assurance review and follow-up, we recommend that any supporting documentation, such as screen-prints, where appropriate, be identified to link them to the specific test case.

Electronic screen shots are also sufficient.  If desired, these can be printed and included as part of the testing process.  You test methods should be defined in a SOP.

We recommend that you test at your peak production times and with the maximum number of concurrent users to assure proper system performance.

Obviously, this type of testing may not be possible for systems with a large number of licenses.  A representative production environment with multiple end user input (if required) is sufficient.

We recommend that you test to assure you have correctly defined system security and that all users can log on with the correct security privileges.

A range of end users with a variety of access privileges should be included as part of usage testing. Again, it is not necessary to confirm log on access of all users within the system.

We recommend that you validate all interfaces, including those to a Hospital Information System (HIS) (e.g., admissions, discharges and transfers (ADT)); a Laboratory Information System (LIS) (e.g. order entry/results return (OE/RR)); and all instrument interfaces, as applicable. Monitoring of interface error files should continue subsequent to implementation as unforeseen transaction types or data elements may cause disastrous results (such as orders not being received, results posting to the incorrect donor/patient record, etc.)

Frequent system audits should be performed as part of the integration of any new system.

• You must establish procedures for the proper use of your system (21 CFR 211.100(a), 606.100(b)).
• You must train personnel in the use of the system procedures (21 CFR 211.25(a), 606.160(b)(5)(v)). Training should include assessing the user’s ability to understand and correctly use the system. You should evaluate the ability of computer operators to perform system maintenance procedures, such as backups, and their ability to respond in an appropriate and timely manner to all alarms, warnings and error messages.

Training should be defined for the appropriate level of end user.  Not all users will need to know how to backup the system but all users should know basic system warnings and messages.  Again, system usage is the key.

We recommend that you validate or verify the output of system reports. We recommend you include any reports containing donor or recipient history information, product quarantine reports, donor or patient merge reports, and patient chart reports (as applicable) in this effort. It is particularly important to ensure that reports print in lower and upper case if the system keeps records of red blood cell phenotype or antibodies.

All reports which are utilized to make any patient or manufacturing decision are critical and thus require validation.

We recommend that you closely monitor the system for a period of time after installation of the new or upgraded software in the production environment to detect any discrepancies that were not identified by the test cases.

A Deviation, CAPA, and Change Control system must be in place prior to system release.

If your Blood Establishment Computer System includes the use of wireless radio frequency (RF) technology, we recommend that you evaluate the device for wireless coexistence, performance, data integrity, security, electromagnetic compatibility (EMC), and electromagnetic interference (EMI) in the setting in which you will use it. For further information, see the draft guidance document entitled, “Guidance for Industry and FDA Staff, Radio Frequency Wireless Technology in Medical Devices” (Ref. 9). When finalized, this draft guidance will represent FDA’s current thinking on this topic.

This topic will be addressed in a future post.

Although not a function of your Blood Establishment Computer System, we recommend that you validate the data conversion from your legacy system to your new BECS to avoid problems such as duplicate donor records and deferral codes.

 Again, a full system audit should be performed prior to release into the production environment.

 

1. Validation Report

You must document your validation activities pursuant to 21 CFR 211.68, 211.100(b), and 606.160(b)(5). Your validation report should include a summary of the test results, including any variances or failed tests, any amendments made to the test cases or the validation plan, an evaluation of the outcome of your testing, and approvals or sign-offs by management.

 

I recommend that management be involved early in the process to avoid delays during the approval process.  At a minimum, input should be gathered during the creation of User Requirements Specification (URS).

 

2. Validation after a Change

Due to the complexity of Blood Establishment Computer Systems and BECS, a seemingly small local change may have a significant global system effect. When any change (even a small change) is made to the software on the Blood Establishment Computer System, a validation analysis should be conducted, not just for validation of the individual change, but also to determine the extent and impact of that change on the entire system. Based on the analysis, you should then conduct an appropriate level of software regression testing to show that unchanged but vulnerable portions of the system have not been adversely affected. Appropriate regression analysis and testing provide the confidence that the system is validated after a software change (Ref. 2). We recommend that you perform regression testing, when indicated by your analysis, by re-running test cases that previously passed.

When the BECS manufacturer provides modifications to the software, the manufacturer should inform you of other functions that might be affected. We recommend that your contract with a software manufacturer address this. However, you are ultimately responsible for assuring that the Blood Establishment Computer System used in your establishment has been validated for its intended use.

 

The validation analysis should include a Risk Assessment.  This assessment will determine the level of testing required, if any.

 

3. Integrated Package vs. Stand-Alone

If your Blood Establishment Computer System or BECS is part of an integrated package on a Laboratory Information System (LIS) or Hospital Information System (HIS) rather than a stand-alone system, you should also consider how changes made to functionality shared by the LIS or HIS and BECS (such as ADT, orders, etc.) might affect your BECS. You should determine what functions or files your LIS or HIS and your BECS share by discussing this with your BECS manufacturer.

Testing should be conducted along the system boundaries but may include data input from a separate system.  It is better to separate the systems along the system boundary lines and perform the required testing accordingly even if this mean duplicate testing of inputs / outputs. 

4. System Documentation

You must maintain documentation for the Blood Establishment Computer System (21 CFR 211.68, 211.100(a), 606.160(b)(5)). You should ensure that the documentation is current, accurate, and as detailed as necessary to ensure proper use and operation of the system, and ensure that all components that should be validated are validated. We recommend you include, but not necessarily limit this to, the following, as applicable to your blood establishment:

Items Supplied by the Software Developer, such as:

•  
  • hardware specifications and hardware requirements
  • instruction manuals, e.g., User’s Manual, Operations Manual, Installation Manual, System Administration Manual, etc.
  • environmental requirements

Items Supplied by the User, such as:

•  
  • system description
  • network diagram
  • list and location of peripheral devices such as printers and terminals
  • processor type(s)
  • operating system and version number
  • system memory
  • disk configuration
  • type and location of backup media
  • list of interfaces
  • list of environments on system, e.g., test, training, production, etc
  • SOPs
  • validation records
    • record of hardware and software maintenance, including date performed
    • record of changes made to system hardware, software and peripheral devices, including date
    • user training records
    • problem reports and their resolution

Note: Although a remote access log is not part of validation records, you may consider keeping a log of remote access to any part of your system, such as the BECS or instruments. We recommend this log include the date, time, reason for the access and the name of the person connecting to your system.

A system log is a critical component.  This log should be maintained by IT or system administrator and include all activity performed.  In addition, all IT /system administrators should be trained on the principles of software validation and change control to avoid activity that may affect the validated state of the system.  Any major changes should be approved by the system owner prior to implementation.

 

 

I welcome any and all feedback on my posts.

 

If you are interested in how my expertise can contribute to your validation project or company, please contact me at greggsalomon @ yahoo dot com.

System Configuration Documentation: Don’t release to production without one

System Configuration Document: a key component of validation that should be in place before system release

 

Overview

 

A System Configuration Document captures the final state of configuration of the system being validated.   This state should be reflective of the initial usage environment that will be in place at the time of system release.   Some of the key areas to be aware of are as follows:

• Timeframe
• Reflection of Usage
• Level of Detail
• Change Control
• System Restore
• Audit Impact

This post is relevant to anyone performing an equipment or system validation.

 Timeframe

 When should a System Configuration Document be completed?  Some might argue that it should be developed in parallel with the User Requirements Specification (URS) after initial system identification or development.  Others would argue that it should only be in place prior User Acceptance Testing (UAT) / Performance Qualification (PQ) testing.  The bottom line is that is should be in place once the system is developed/identified and the URS is approved.  At a minimum, a draft version of the System Configuration Document should be in place prior to any usage testing.  This document will be utilized to provide the defined configuration of the system necessary for testing.  It can and should be updated to reflect the outcome of usage testing as necessary if issues are discovered with the current configuration.

DO NOT wait until after release of the system to create this document.  Further details of why this is not appropriate will be discussed during the Change Control section.

 Reflection of Usage

It seems pretty obvious that the System Configuration Document would reflect the actual usage of the system.  It is important to mention here that accuracy is the key to a success.  Make sure that the System Owner is involved in this process as a final check to confirm the document matches what is current within the system.  It may be necessary to include other individuals within the process for a complex system with boundaries that may impact other systems or groups (i.e. IT Staff, Engineering, QC/QA, system SME, system vendor, etc…). 

 

 Level of Detail

 My opinion is the more details, the better.  It will take a bit more effort up front to capture all the details of the initial configuration of the system but it is critical information that can be utilized during change control, system restore, and system audits.  Include diagrams, screenshots, tables, and pictures as necessary.  The document should allow a user with basic system knowledge to reproduce the configuration of the system.

 Change Control

 A mentioned earlier Change Control is an important reason for the proper creation of a System Configuration Document.  Change Control captures all changes to validated systems.  A System Configuration Document provides the initial starting point or baseline of the system.  It should be possible to track all changes to a released system back to the System Configuration Document.  The System Configuration Document can be formally revised once the system is released to reflect the current state of the system per Change Control.

 System Restore

 Another important and overlooked function of a System Configuration Document becomes apparent when it is necessary to perform a system restore.  This easily applies to software but should also apply to equipment if it becomes necessary to replace equipment or a system due to disaster or other reason.  A well-written document can help to restore the system to its previous configuration in a much shorter timeframe with a high level of confidence and minimal production disruption.

Audit Impact

 Audits are perhaps the main reason for proper System Configuration Documents.  Effective System Configuration Documents show an auditor that quality planning were part of system validation.  Change Control is often a central part of any system audit.  As stated earlier, it is easy to trace the history of the system back to system release with a quality System Configuration Document.  And with an updated System Configuration Document, it is easy to show the current configuration state.

 I welcome any and all feedback on my posts.

 If you are interested in how my expertise can contribute to your validation project or company, please contact me at greggsalomon @ yahoo dot com (this format is to prevent auto spammers).

Audit Vendor Validation Documentation before purchase..or get the shredder ready.

Audit All of your Vendor’s Validation Related Documentation before Purchase

 

Overview

 This post is the result of a number of complaints from my clients who purchased systems from vendors and validation related documents, templates, protocol, etc for these systems.  These validation documents (and validation services) were sold either separately as line items, or as part of the system (with the appropriate discount incentives added).  The bottom line is that a complete quality audit was never done on these documents by the clients prior to the purchase.  In the case of the particular vendor in question, (I won’t name in this post) the documentation was not shown to the clients prior to purchase with the excuse being that if the documents were audited they could be reproduced without purchase.  The vendor assured the clients that all documents were developed by Quality Professionals with years of experience within the industry.  Having seen these documents, my opinion is that they belong in the shredder.

 Vendor Audits

 A Vendor Audit should always be done prior to purchase of any system.  In this case, all validation related documents should be audited prior to purchase.  There is no threat to the vendor of a client using their validation documentation without payment if the client performs the audit in person at the vendor location (or the vendor representative can provide these documents in person at the client site).  Validation related documentation can offer a window into the Quality System utilized by the vendor (if any).   

 Vendor Quality Process

 This is a vital part of the selection process for any vendor.  The overall audit of the vendor should include how validation documents were developed and what procedures are in place which governs the development of the documents.  This should be part of the overall vendor/supplier audit.  Even if it is simply an audit of the validation documentation and services, it is important to confirm the quality of the deliverables before purchase.   Smaller companies and organizations may rely upon the vendor for advice and guidance.  In this case, it may be well worth the expense to hire a quality professional or utilize a contractor with validation experience.  Ignorance is not an excuse that one can give during an audit or inspection.

 Qualified Service Delivery

 In the case where a vendor offers validation related services, make sure the people performing the service are qualified from a technical and quality perspective.  A technical person lacking quality experience is acceptable if the client wishes to oversee the quality requirements and has a strong quality system.  Keep in mind that this may increase the risk of an observation if close attention isn’t paid to the work performed or tempaltes/protocols utilized.

 Flexibility

 A Vendor offering validation templates, documents, protocols, etc. should always be willing to adapt their methods to the client’s Quality System.  This is CRITICAL!  A vendor isn’t responsible for the validation documentation and services during an audit/inspection.  The responsibility lies solely with the client (responsibility can never be outsourced).  In some cases the vendor may be the expert from a technical standpoint, but the client should have full control of their quality system and regulations/requirements that they must adhere to.  In addition, there may be existing policies and procedure already in place that may conflict with the vendor provided documentation or services.  If the vendor isn’t willing to adapt their services and deliverables to the client, there will most likely be more work ahead to correct the issues encountered.  More work equals more time and money.  A vendor should be more than willing to expose their Quality Systems and practices to clients as a method of reassurance.  Any vendor who hesitates to expose their Quality System is most likely trying to hide significant issues.

 Audit versus Time, Effort, Money, and Risk

 Will the cost of performing an audit up front be more than the time, effort, money, and potential risk if the quality of the validation documents and services does not meet expectations?  What happens if the audit of a vendor determines their quality systems are substandard before or after purchase?  Does substandard quality in their validation related documentation reflect on their product, equipment, or system?

 These are all questions that should be in the back of your mind.  

 I welcome any and all feedback on my posts.

 If you are interested in how my expertise can contribute to your validation project or company, please contact me at greggsalomon @ yahoo dot com.

Dangers of using ‘deviations’ during a validation

via flickr by megapiksel

Never report incidents as ‘Deviations’ during a validation

 

   This is a common mistake encountered during a validation when an error is discovered in the system, method, equipment, software (or whatever item is being validated) is that the incident / error is documented as a ‘Deviation’.  I can’t stress this enough…you should NEVER write a deviation during a validation.  Only in a single instance would I consider using a deviation during this process which I will discuss as part of this post.  This post is relevant if a validation is being performed and for those companies, organizations, and consultants learning about the proper way to perform a validation for a quality perspective.

 

Case Study of Software Validation:  Why deviations are a bad idea during validation

 

While working for a previous employer during a software validation for a State-run laboratory, I was required to document all errors encountered during the process using a combination of deviation forms and their deviation process.  I was unsuccessful in my repeated attempts to explain the Risk involved with capturing all errors within the client’s deviation process.  Unfortunately, the EDMS software vendor  had some OQ protocols that were poorly written (this incident finally convinced them to improve their quality process) coupled with the client’s strict interpretation of deviations, led to over 100 deviation entries into their production system and 50-75hours worth of downtime processing these records.  If I was an inspector, this would definitely raise a red flag.   I told them that when they utilize their deviation system (within their live, production process); they will encounter the following:

 

1)      It artificially increases the importance of the encountered errors beyond what they should be for a non-validated system.

2)      Adding non-validation systems to the deviation process greatly increases the overall number of deviations within the system.

3)      Large delays occur during the validation as the deviations are handled in a time consuming, formal process versus an incident report process.

4)      Deviations are a common inquiry and therefore will lead inspectors directly to the validation process for the system which might have otherwise been ignored during the audit/inspection.

5)      Questions arise about the system selection process if there are many formal deviations during validation.

6)      It greatly increases the cost of the validation project when formal deviations are entered into the system under strict guidelines versus an informal incident process which typically takes less time to process.

 

Costly Outcome

 

The overall outcome was a significant delay of the validation process, and a huge cost in terms of employee hours from IT, QA, System Owner, and client personnel involved in testing process.  Because of the lengthy and formal deviation process, the number of individuals involved in the process was typically five or more, including myself, System Owner, IT, QA, and the client personnel assisting me (and some meeting that were required to close the deviations).  Calculating all the individual’s salary costs and our per hour costs, at least 10K was wasted during this process.  Sadly, this might not be the most costly part of the project.

 

Regulatory Impact

 

The regulatory impact of entering deviations encountered in the process for non-validated system may be more costly that the immediate monetary impact.  As stated earlier, many validations entered into the deviation system will flag that system as a potential problem (even if these are merely validation related errors).  Again, deviations always targeted by inspectors from regulated agencies, your regulated customers, and your internal QA audits.  The best ways to avoid these issues are to report system errors during validation as ‘Incident Reports’ instead of deviations.

 

Incident Reports

 

Incident Reports are segregated from the deviation process.  Incident Reports are similar to Deviation Reports where the issue is identified, root cause discovered, and issues documented and resolved.  Incident Reports differ in the way these are resolved versus Deviations.  Typically, Incident Reports do not require the same level of documentation or formal process to address.   Errors encountered as part of the execution of validation testing should always be handled using this method.  There is never a need to include this type of issue within the deviation process for a non-validated system.  I have yet to be a part of a validation where issues did not occur. In fact, it is not only typical but expected to see many more incidents early on during the validation process when the ‘bugs’ are being worked out of the system.  The one exception to handling these issues in Incident Reports would be if there is a need to deviate from the validation process in terms of already approved procedures.  For instance, if a different validation method was to be utilized or a new protocol was to be issued during validation, a deviation against the process would be justified.  Again, this would best be positioned against the validation procedure, not the specific system being validated.

 

 Segregation of Validation is Vital

 

Once again I want to stress the importance of keeping your validation documentation separate from your ‘live, production’ system.  There is no need to make an auditors life easier by giving them a clear path into your validation process via your deviation system.  Equally, you should not spend all the time, money, and effort on performing a proper validation only to have the efforts questioned because of a simply lack of quality process understanding.  Keep Incidents Report contained within each validation project and make sure that deviations are NEVER part of your validation process.

 

If you are interested in how my expertise can contribute to your validation project or company, please contact me at greggsalomon @ yahoo dot com.

21 CFR Part 11: Does this apply to systems with both electronic records and paper format?

 

 

via flickr by wokka

 

Overview

 

This has become a topic of discussion within several LinkedIn groups.  Many people adamantly argue that their system doesn’t fall under part 11 requirements because they maintain a hybrid system of electronic records and paper printouts.  This argument may be true depending on the types of records and how they are utilized within the process.  The FDA has already released a guidance document for industry on this topic http://www.fda.gov/CDER/guidance/5667fnl.pdf .  I will discuss the important issues within the above listed guidance document in my post.  This post is applicable for any company with questions about part 11 or utilize a hybrid electronic and paper based system of records that fall under predicate rules.  

 

 

Categorization of Electronic Records

 

Not all electronic records are subject to part 11.  The FDA has states, ‘that Part 11 applies to records in electronic form that are created, modified, maintained, archived, retrieved, or transmitted under any records requirements set forth in Agency regulations. Part 11 also applies to electronic records submitted to the Agency under the Federal Food, Drug, and Cosmetic Act (the Act) and the Public Health Service Act (the PHS Act), even if such records are not specifically identified in Agency regulations (§ 11.1). The underlying requirements set forth in the Act, PHS Act, and FDA regulations (other than part 11) are referred to in this guidance document as predicate rules.’ Any records that not do not fall under the predicate rules and are not part of a submission, are not subject to part 11 requirements.  Make sure to define which systems and/or records fall under the predicate rules to determine if further action is required.

 

 

Hybrid Systems

 

This is where most of the confusion comes into play.  If a system has both electronic records and paper format, part 11 requirements come into play only if the electronic records are relied upon to perform regulated activities.  The FDA states,’ In some cases, actual business practices may dictate whether you are using electronic records instead of paper records under § 11.2(a). For example, if a record is required to be maintained under a predicate rule and you use a computer to generate a paper printout of the electronic records, but you nonetheless rely on the electronic record to perform regulated activities, the Agency may consider you to be using the electronic record instead of the paper record. That is, the Agency may take your business practices into account in determining whether part 11 applies.’  Basically, if you have an electronic system with paper records, make sure you don’t utilize any electronic data (including reports or metrics) within your quality process especially if the paper system was instituted to avoid performing validation.  You might just end up with a compliance issue and have to validate the system anyway.  Of course this does not apply to word processing or e-mail but make sure you’re careful and use common sense.  If you’re still not sure, seek professional advice.

 

Define Your Process

 

To keep things simple and to avoid confusion, determine which format will be utilized for all records that fall within the predicate rules.  For hybrid system, it is particularly important to define the use of these records within a SOP or specification document.  If necessary, perform a system audit to determine exactly how the process takes place and how the data is constructed, analyzed, reviewed, approved, reported, and maintained.  Determining this information in advance will avoid potential issues during an audit.

 

 

I welcome any and all feedback on my posts.

 

If you are interested in how my expertise can contribute to your validation project or company, please contact me at greggsalomon @ yahoo dot com.

 

 

 

 

Warning about Linkedin Validation Consultants

Validation Consultants on Linkedin

via microsoft clip art

 

I’m writing this quick post because of a distrubing trend I’ve discovered having joined many validation related groups within Linkedin.   I’ve noticed that there have been many groups that are utilized by consultants for purposes of generating revenue for their validation business.   The worrisome trend is that these individuals who start these computer system validation groups are taking advantage of the lack of knowledge of people within the regulated industry seeking advice.  Even today, I left a group after realizing that the group moderator was giving incorrect advice to the readers on clinical computer sytem validation and insisted on his point of view even after I tried to correct his error.  For the sake of your own quality system and the safety and quality of products/services, please be aware of the ‘experts’ on Linkedin and do your homework before hiring any of these consultants/companies.

Kind Regards,

Gregg Salomon

Risk Assessments for Computerized Systems in the Blood Industry

Risk Assessment for Computer System Validation in Blood Establishments

 

via MS Clipart

 

Overview

 

Having previously performed numerous Risk Assessments on Computer Systems for many Blood Centers, I wish to share the importance of performing a proper Risk Assessment.  Now that I am an independent consultant and no longer limited by prior employer policies and politics, often driven from a cost (not quality perspective), I can freely state the correct methodology for Risk Assessments.  The main focus for a Risk Assessment should be to define the level of testing required for system validation based upon the risks to patient and quality of product.  It is imperative to perform Risk Assessments early in the process to help define the validation activity before the validation plan or test cases are developed.  Risk Assessments should be utilized throughout the process and post implementation (separate or integrated within Change Control).  My recommendation is to utilize a worksheet to capture this process and thus established a specific timeframe.  Further information on this topic can be attained from http://www.fda.gov/cber/gdlns/becsvalid.htm (FDA/CBER guidance doc) and http://www.ispe.org/cs/gamp_publications_section/gamp_publications_overview (GAMP5 publication).  This post will be relevant for companies within the blood industry (and most other regulated industries) for compliance with computer system validation.

 

 

Define Critical Quality Attributes

 

Critical Quality Attributes (CQA) are central to developing a proper Risk Assessment.  CQA are best defined as those attributes that have the greatest impact upon patient safety and product quality.  Once these are defined, specific system standards and requirements can be developed which further define testing methods and risk focus.  This is the basis for making science and risk based decisions that will ensure that the system is designed or configured and meets its intended use.   Make sure that as part of this CQA definition, your regulatory requirements are indicated. 

 

 

Risk Assessment

 

I cannot stress enough how important a Risk Assessment is for proper system validation.  Once the basic system requirements of the system have been identified, this Risk Assessment is done to help define the validation plan and what validation test will be required for the system.  Without doing a proper Risk Assessment, key CQA areas might be omitted or unnecessary testing may be done.  One important thing to keep in mind is that the total elimination of all risk is not the focus; it is justification of residual levels of risk.  Risk may simply not be able to be eliminated or the cost associated with total elimination may not be justified by the business.   Risks identified may change over the course of the system validation.  It is possible to perform ongoing Risk Assessments during the course of validation as additional items are identified.  I have found that the best method for capturing this information is to utilize a Risk Assessment based worksheet.  This worksheet should capture the following information:

 

• Initial Risk Level Classification
• Identified Issue (what is causing the risk)
• Identified Risk Component(s) (specific risk or risks from the Identified Issue)
• Likelihood of Detection
• Likelihood of Occurrence
• Severity of Impact
• Mitigation Strategy
• Corrective Actions
• Final Risk Level
• Justification of Final Risk Level

 

In circumstances where the Severity of Impact is high, assume that the Likelihood of Occurrence is high if there is no way to determine this outcome.  This should be noted in the Final Validation Report and increased auditing of the system should occur until a level of confidence is established.

 

Frequency of Risk Assessments

 

To keep things simple, this is the basic guidelines for when to perform a Risk Assessment:

• Prior to the start of validation but after the System Requirements have been defined
• During the course of validation, especially when changes to the process occur or when additional issues are discovered that would warrant a Risk Assessment
• Prior to implementation of the system when all testing has been completed
• Periodically during system audits
• During every change to the system via Change Control or as a separate Risk Assessment
• Prior to decommissioning of the system

 

 

I welcome any and all feedback on my posts.

 

If you are interested in how my expertise can contribute to your validation project or company, please contact me at greggsalomon @ yahoo dot com.

 

Validation Documentation Errors (grammar/spelling): Correct them?

Reader Question:  Is it worth correcting errors within validation documentation if it is technically correct?

 

spelling-via-flickr-by-rathianspkie

 

  

 

 

Overview

I received an email from reader of my blog who has hired a validation consultant to write part of his validation documentation but received deliverables that were less than ideal.  Here is the actual email:

 

I would be grateful if you would provide an opinion on this matter: Within a project group, the project manager has assigned a developer to author user requirement specifications (after interviewing the users) and functional specifications. The project is modular, so there are many specifications. The specifications typically contain grammar errors, as English is not the author’s native language.

What is the risk presented by a specification that is technically correct but grammatically of poor quality? Is it worth the effort to correct every sentence if the meaning is discernible, even though the language may be inelegant?

 

 

 The short answer is ALWAYS correct the errors.  I’m not saying that the entire document has to be revised for a few minor spelling mistakes.  In the following sections I will tell you why.  This post is relevant to companies who have hired validation consultants or companies to perform part of all of the validation activities.

 

Compliance Issue

 

The first major concern would be the potential issue of a grammatical or spelling error that causes an actual compliance issue.  If the meaning or intent doesn’t reflect what actually happened, an astute auditor will pick up on this error and require clarification or consider the test as invalid.  This observation will most likely occur months or years from the actual validation event and may be difficult for someone to reconstruct even if the most likely event is obvious to the System Owner or Quality Reviewer (assuming that those people haven’t left the company).

 

Proper Review

 

Technical errors may not be discovered if an auditor is not a subject matter expert in your system.  With grammatical/spelling errors, almost anyone can identify these issues without any system knowledge.  Furthermore, all of these documents require a quality review and approval from the quality unit of the company, not the consultant (remember responsibility cannot be outsourced even if the validation tasks were).  The assumption is that the person performing the review of the validation documentation is a native/expert in the English language (and it should be) and that the errors should have been identified and corrected as part of this review process.  If I was the auditor, I would assume that these documents were given a ‘rubber stamp’ review.

 

GDP and Document Control/QA

If there are documents presented to auditors that are in obvious need of revision, this may lead to further questions on the company’s understanding of Good Documentation Practices (GDP) and Document Control/QA practices.  A good auditor will take this opportunity to dig into the GDP which may lead to further questions for Document Control and QA.  Additional question may arise regarding the validation of other systems, review policies, and other documentation within your company.  At a minimum, the auditor may spend additional time reviewing the validation documentation of the current system due to these initial findings.  

 

Lack of Understanding

 

Another concern I would have when selecting a consultant who may be technically savvy but grammatically challenged is their potential lack of understanding.  If they cannot product a well written document, do they actually have a proper grasp of the language?  I’ve worked with many different consultants from countries who have a significant understand of the language and are technical experts in their fields. 

 

On the other hand, I’ve worked with some individuals who were able to BS their way through the vetting process and clearly do not understand the language well enough to function as expected.  What makes this worse is that they will usually not admit the misunderstanding if confronted but rather state that they can do the work and understand what is required.  The last thing bad a consultant will admit is that they don’t understand the request.  This is paramount to a dismissal in their mind.

 

Don’t Sacrifice Quality

 

In the realm of quality/compliance related documentation, you really don’t want to sacrifice quality.  As the person who sent in the request has probably already discovered, it cost time and money to fix a problem that could have been corrected up hiring a person with a solid grasp of the language (and understanding of validation/quality). 

 

It happens to all of us in this current economic environment when every dollar counts.  My previous company hired workers from overseas to create software and validation PQ templates related to the software.  It turns out that the templates were loaded with errors resulting from an obvious lack of understanding of the requirements and therefore required additional re-work before utilization.

 

Ask for a writing sample, contact previous references, and conduct an interview prior to hiring a consultant to perform any validation activities.  If you are only concerned about costs, you’ll often get what you pay for.

 

Finally, validation testing doesn’t have to be perfect.  There have been many incidents in the systems that I’ve tested which were captured and well documented.  Auditors aren’t expecting that a new system will be perfect right out of the box.  Validation is the time where these issues should be discovered and corrected by following sounds policies and procedures.  However, a fully controllable event like validation documentation should be as close to perfect as possible to show control over the process.

 

 

I welcome any and all feedback on my posts.

 

If you are interested in how my expertise can contribute to your validation project or company, please contact me at greggsalomon @ yahoo dot com.

Software: medical device or aid/tool?

FDA guidance:  Is your software a medical device?

 

via flickr by john-norris

 

 

Overview

I’ve recently been in contact with Mr. Joseph Tartal, Technical Branch Chief, DSMICA, CDER, FDA, in regards to comments I had on the 2007 Guidance for Industry Computerized Systems Used in Clinical Investigation. The topic discussed was clinical software and its status as a medical device versus a tool or aid.  This post is relevant to companies who plan to create or utilize critical software within their regulated operations.

 

 

2007 Guidance for Industry Computerized Systems Used in Clinical Investigation

This is the 2007 Guidance which covers computerized systems used within clinical trials.  I provided some feedback on this guidance to Mr. Tartal on the related documentation that should be part of implementation of this type of system (see my previous post).  The feedback I got led to discussion of when a computerized system becomes more than just a tool or aid.  The next section is the actual e-mail response from Mr. Tartal.

 

 

Mr. Tartal’s e-mail response March, 6^th 2009

 

 

Dear Mr. Salomon,

 

Clinical computer systems, both utilized in clinical trails and to make health care decisions, are becoming a lot more prevalent and are expanding with regards to applications and data management.  They are no longer just record retrieval systems.  As this software evolves so does their status as medical devices.  One of the terms currently in the mix is diagnostic software; where does the line exist between a tool or aid and a medical device?  What are the requirements as they apply to software as a medical device: lots of guidance has been put into place to help explain this.

http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfTopic/topicindex/topindx.cfm?alpha=s (Topic Index: Software Guidance)

 

If you have specific questions on the guidance I would recommend that you contact the centers software expert, John Murray, at (240) 276-0284.

  

Thank You,

Joseph Tartal, Technical Branch Chief

DSMICA

This response represents to the best of my judgment how the device should be regulated, solely based upon a review of the information you have provided. This communication is consistent with 21 CFR 10.85 (k) and constitutes an informal communication that represents my best judgment at this time but does not constitute an advisory opinion, does not necessarily represent the formal position of FDA, and does not bind or otherwise obligate or commit the agency to the views expressed.

 

 

Again, it is important to state that this was informal feedback from an e-mail exchange and does not constitute an advisory opinion, formal position of the FDA or bind or otherwise obligate or commit the FDA to the views expressed.

 

 

Software as a medical device

Although the comments from Mr. Tartal were related the use of software within the clinical environment, I believe that this reach will extend beyond the realm of diagnostic tools within the clinical environment to cover other areas where quality related decisions are made by software systems.  The question is at what point does the computerized system become more than a tool or aid?

 

 

I welcome any and all feedback on my posts.

 

If you are interested in how my expertise can contribute to your validation project or company, please contact me at greggsalomon @ yahoo dot com.

Documents recommended for Clinical Software implementation:FDA 2007

 Documents recommended for Clinical Software implementation:  FDA 2007 guidance

by nacaseven via flickr

 

Overview

 A common question I get from my clients is ‘what documentation/SOPs are necessary to properly implement a regulated computer system?’  I recommend a list found in  Appendix A of Guidance for Industry Computerized Systems Used in Clinical Investigations FDA 2007 Guidance Document http://www.fda.gov/Cder/Guidance/7359fnl.pdf.   This list is defined for Clinical sites and related activities as stated in the scope of the document. However, after reviewing the list provided in Appendix A, I believe that it is applicable to other regulated systems beyond the scope clinical systems.  This post is recommended for anyone who plans on implementing a regulated computer system.

 

List of Documentation

 

This is the actual verbiage from Appendix A.  I’ve added my own comments below in BOLD.

 

Standard operating procedures (SOPs) and documentation pertinent to the use of a computerized

system should be made available for use by appropriate study personnel at the clinical site or

remotely and for inspection by FDA. The SOPs should include, but are not limited to, the

following processes.

 

 

• System setup/installation (including the description and specific use of software,

hardware, and physical environment and the relationship) I would include Configuration Document/Procedure to specifically describe how the system is setup.  Screenshot within this document are a strong recommendation.

 

 

• System operating manual  The software/hardware vendor should provide this but make sure you review this once you understand how the system operates.  Often times the vendor will not make the proper corrections between software releases and may lead to improper software operation if this is the primary end user training.

 

 

• Validation and functionality testing See GAMP5 or other CSV recommendation.  This is too big of a topic to comment on within this post.

 

 

• Data collection and handling (including data archiving, audit trails, and risk assessment) Risk Assessment should be its own document as far as system Risk is concerned.  It may also be included as part of Change Control procedures.

 

 

• System maintenance (including system decommissioning) I would make the system decommissioning a separate process and include a Data Retention procedure.  Make sure you can still access stored information for the appropriate amount of time after the system is decommissioned.

 

 

• System security measures  Don’t forget to coordinate with the overall IT procedures on system security.  You may be out of compliance if your program security procedure is separate from IT computer security and they don’t follow the same security requirements. (i.e. password expiration, length, login attempts before user lockout, etc.)

 

 

• Change control I would personally limit Change Control to validated systems.  You may encounter many errors during validation that are a normal part of the validation process that may otherwise overwhelm your Change Control process and delay system validation.  However, process/procedure/quality related changes may still require Change Control.

 

 

• Data backup, recovery, and contingency plans  Don’t forget to test the system recovery as part of this process.  Having the procedure is fine.  Make sure the backup and restore actually works before you actually need a real restore of your data.

 

 

• Alternative recording methods (in the case of system unavailability)  A paper backup is one method that is possible.  Another method would be duplication of the production site, alternative hardware backup (server or other HW), duplicate system.  IT can be a valuable resource for alternative methods or configurations that will mitigate risk.

 

 

• Computer user training The vendor will often provide material related to system usage.  Make sure this material is specific to the level of access for the end user and their associated job function.  In addition for COTS applications, the specific configuration may require extensive revision of the vendor provided training.

 

 

• Roles and responsibilities of sponsors, clinical sites and other parties with respect to the

use of computerized systems in the clinical trials  One area that should be addressed is Signature Authority.  If a user delegated their authority to another user per procedure, make sure the software can capture this information properly.  Another recommendation is a roles/rights matrix for the system users.  It is better to define this per role in the system if possible since this matrix will have to be updated with personnel changes if it is user based.

 

 

 

 

 

I welcome any and all feedback on my posts.

 

If you are interested in how my expertise can contribute to your validation project or company, please contact me at greggsalomon @ yahoo dot com.