News Out-of-specification results
Here you will find answers to the following questions:
|
14.H.1 Significance
Ever since the groundbreaking supreme court ruling in the USA (FDA) against BARR Laboratories (BARR, 1993), the area of "Test results outside the defined specifications" (OOS = Out of Specification) has become the central aspect of each official inspection, particularly if carried out by the FDA. Regular publication by the FDA of the results of inspections carried out worldwide (GMP Trends, Gold Sheet, Warning Letters) clearly demonstrates time and again that in this area grounds for complaint are commonplace.
As a general rule, it must be assumed that the quality assurance system used during production guarantees the quality of a given product (validated manufacturing procedure). The quality control department only checks the quality of the product with reference to the set (registered) specifications. The result of these analyses leads to the approval or reworking/rejection of the batch concerned. The FDA guidelines in this regard are exclusively oriented towards the quality control laboratory. The aforementioned check is therefore of central importance as a test result deviation can provide important information on (possible latent) defects in the quality assurance system. The correct investigation of these kind of results is ultimately a measure of the company's GMP awareness. And last but not least, the approval of a batch for use by people (with diseases) where at best the quality is questionable is not permissible from an ethical point of view.
14.H.1.1 The BARR Laboratories case
Following several inspections by the FDA since 1989, BARR Laboratories voluntarily reduced the number of products sold from 175 to 115. However, further measures by the company were initially not accepted. In the aforementioned court case (BARR, 1993) (Schmidt, 1998), the decision not to grant approval of an additional 24 products as well as to recall 12 different product batches based on the FDA results has been upheld. In addition to further deficiencies in the area of quality control, the authorities accused the pharmaceutical manufacturer of not exercising the required care or scientific approach particularly when investigating OOS results. Apparently, analytical results that conform with the specifications were obtained simply by carrying out continuous repeat measurements that led to the batch being released. The FDA stated that, in their view, this procedure of "analysis into compliance" was unacceptable and demanded instead that fault classification be carried out for
- laboratory errors
- errors not related to the manufacturing process
- errors related to the manufacturing process
as well as one systematic failure investigation in the laboratory according to scientific criteria. The requirements were commented upon by Paul F. Vogel (director of the manufacturing and product quality department at CDER/FDA) (Vogel, 1993) which finally resulted in the Guide to Inspections of Pharmaceutical Quality Control Laboratories (FDA, 1993).
It therefore appears that the requirements of the FDA are clear but in practise however, their implementation regularly seems to be the subject of debate. Detailed specifications by the FDA were not provided on exactly how an OOS result should be clarified (guideline missing) - for this aspect alone, a serious and scientifically-based procedure was required to clarify the cause. The measures stated in figure 14.H-1 reflect the main points of these investigations.
| GMP-conform handling of OOS results |
|---|
|
14.H.1.2 The consequences
The remarks by Paul F. Vogel made during a forum in September 1993 on re-testing and laboratory investigations (Vogel, 1993) and the Guide to Inspections of Pharmaceutical Control Laboratories (FDA, 1993) form the basis of guidelines for dealing with OOS results (OOS SOPs) at many companies.
Clarification of out-of-specification laboratory results (translated from (Vogel, 1993)).
He observed that there is general consensus between the authorities and the industry that OOS results .....
- ..... are important indicators that the material does not satisfy the quality requirements
- .... cannot simply be ignored in order to disregard the possibility of a product error without a well-founded clarification.
The FDA also recognises the effects of OOS results on companies in terms of costs, delays and delivery problems but the first priority should be to ensure that the patient receives a faultless product. Furthermore, the FDA is categorically not of the opinion that a result can never be declared invalid as it is perfectly possible that a analytical result does not reflect the true quality of the product concerned in individual cases.
Without hindsight however, an OOS result must always be regarded as being valid until investigations prove otherwise.
- The scope and orientation of the investigations must reflect the situation. If necessary, these must also be extended to include manufacturing and handling errors.
- Depending on the circumstances and the result of the investigation, retesting or resampling may be necessary in accordance with GMP.
- The investigations, the results and the consequences must be fully documented and authorised by the relevant centres (SOPs).
- The necessary conclusions must be drawn from the results of the investigations in order to improve the system and prevent further OOS results.
The FDA takes a critical view of companies that produce an exceptionally high number of OOS results. It is assumed that these companies have general problems with GMP or do not have enough qualified personnel. If the results appear "too good to be true", the FDA would also express doubts suspecting that these OOS results have been discarded, disguised or ignored without proper investigation. This would result in criminal proceedings and judicial clarifications (BARR, 1993).
In September 1998, the FDA published a draft on the subject of clarification of OOS results (FDA, 1998, Volume 3) in which the statements by Paul F. Vogel were expressed in the form of a specific procedure for the first time. This draft is currently the subject of intensive discussions around the world (Renger, 1999) (PhRMA, 1998) and will be examined more closely below.
14.H.2 Definitions
Analytical results in QC are always determined for a partial quantity of the product (the sample). This highlights the importance of proper sampling once again (see chapter 14.A Sampling) as it must be ensured that this partial quantity is representative of the total quantity (specified sampling plan).
The analytical result of a test is therefore obtained through a one-time application of the method as stated in the testing procedure (the calculating algorithms for determining the numerical final results are also an integral part of this method).This inevitably means that the method applied must follow the instructions down to the last detail. The result hereby obtained is initially to be regarded as valid, irrespective of whether it is inside or outside the specification.
| Definitions |
|---|
OOS (out of specification): Result out of specification (Note: replaces previous term deviations from standard). |
OOT (out of trend): Result within the specifications but also conspicuous because it is not in keeping with the trend observed over a longer period of time. |
Repetition of the analysis (retesting): Repetition of the analysis using the same sample or quantity of samples. |
Repetition of the injection (reinjection): Repetition of the injection (in an analytical apparatus) from a test solution that is still available. |
Repetition of sampling (resampling): Repetition of sampling of one batch in accordance with a predetermined plan; this may deviate from the original plan. |
Analytical error: Deviation of an analytical result from the true value due to mistakes made when carrying out the testing e.g. as a consequence of technical problems. A differentiation is made between apparent (reproducible) and non-apparent (unreproducible) analysis errors. The former can be attributed to incorrectly performing the analysis (e.g. errors in documentation, incorrect calculation/evaluation, non-compliance with the conditions, incorrect standard, incorrect initial weight of sample/standard, incorrect dilution, uncalibrated analytical apparatus). The latter can be attributed to previously undetected latent errors (e.g. imprecise formulation of analytical method, unsound method). |
Product error: Deviation of an analytical result from the true value due to insufficient product quality as a consequence of failure in manufacturing. A differentiation is made between errors that are not related to the manufacturing process (e.g. incorrect initial weight, incorrect mixing time, operating error) and those that are related to the manufacturing process (e.g. insufficient validation of procedure, imprecise/incorrect formulation of manufacturing procedure). |
Sample error: Deviation of an analytical result from the true value due to a failure in sample preparation, e.g. sampling, sample mix-up, incorrect labelling, change in quality of samples. |
Unknown error: Deviation of an analytical result from the true value due to an error that cannot be assigned to the first three categories. |
Averaging: The formation of averages from several individual values. |
Outlier test: Declaration of analysis results as invalid using statistical methods. |
14.H.3 FDA Draft Guidance and Comments
The Draft Guidance requirements have on the whole already been applied by many companies. Most of the points it contains are beyond dispute mainly because the requirements of overzealous interpretations in the wake of the BARR verdict have been corrected (Schmidt, 1998). The significant points in this guidance document are presented below and compared with comments and statements from the industry.
| 1. Introduction |
|---|
All results that are outside the specifications or acceptance criteria established by NDA/ANDA, pharmacopoeias and the manufacturer are dealt with within the scope of the draft. This contains analyses for the manufacture of active pharmaceutical ingredients (IPC) and final inspection of active pharmaceutical ingredients, for excipients, for other components and for finished products. |
The scope should be limited to the release specifications for the drug product. IPC results and internal more stringent limits should be excluded (PhRMA, 1998).
The definition of the draft guideline's scope is unclear and also wide-ranging (also includes starting materials, preliminary stages, initial synthesis stages) (Renger, 1999).
| 2. Background |
|---|
Information on fundamental importance of laboratory results with respect to conformity of raw materials, containers and closures, intermediate stages and finished products including stability. |
Critics are requesting limitations to the aforementioned scope (Renger, 1999). In particular, the guideline for in-process controls, stability testing, preliminary stages in the synthesis of active pharmaceutical ingredients or for internal, unregistered, more restrictive specifications should not generally be applied (Renger, 1999).
| 3. Identifying and assessing OOS test results |
|---|
The FDA provisions state that an investigation must be carried out to clarify the cause of each OOS result. Clarification is also required if a batch is rejected to determine the extent to which other batches or products may have been affected by the error. This investigation must be recorded in a report together with the conclusions made and measures introduced. Clarifications must be processed carefully, efficiently, impartially and must also be fully documented and be based on scientific facts. During this process, the validity of the laboratory results should be checked before test solutions are discarded. This means that the same test solution can be used to check for (apparent) analytical errors (note: injection errors in the case of a chromatographic analysis, for example). |
It should be noted that, contrary to the requirement in footnote 3 of the guideline, the application should exclude the OOT results (Renger, 1999).
| A. Responsibility of the analyst |
|---|
The analyst is primarily responsible for obtaining exact results and must be vigilant with regard to potential problems. If the result of the system suitability test is unsatisfactory, all data must be identified and declared invalid. Test solutions should only be disposed of once the results have been compared with the specifications. In the case of obvious errors (spillage of test solution), the analysis must not be intentionally continued. If OOS results are obtained, the supervisor must be informed. |
No objections.
| B. Responsibility of the supervisor |
|---|
In the event of OSS results, the supervisor must proceed with the investigation objectively, promptly and impartially. Possible laboratory errors must be clarified immediately providing that the test solutions are still available (note: and stable). The following steps must be assessed:
He/she must track trends. |
No objections.
| 4. Investigating OOS test results |
|---|
If the initial investigations do not reveal (note: apparent) laboratory errors, this must be thoroughly clarified using a testing protocol. This is a challenge for staff and company management and must be treated as the highest priority. Quality control is responsible for implementation even if other departments are involved. |
Troubleshooting should not have to be carried out by quality control outside of its own area (Renger, 1999) (PhRMA, 1998).
| A. General investigational principles |
|---|
The troubleshooting process must be efficient, factually based and well documented. There clarifications should be documented on a step-by-step basis in the written records as follows:
|
No objections.
| B. Laboratory phase of an investigation |
|---|
The following options are available for this investigation:
The same sample that was originally taken must be used during retesting (cf. sampling) and the sample preparation (initial weighing) is to be repeated. The analysis is to be carried out by another analyst. Repetitions may not be carried out ad infinitum and the number of retestings must be determined beforehand. Once the OOS result has been confirmed, the batch must be rejected. If the OOS result is disproved, the first result is to be declared invalid and replaced by the new one. |
In exceptional cases, repeat sampling may be indicated if the first clarifications demonstrate a large spread of different aliquots. Other test schedules can also be approved. |
Averaging is only carried out in special cases, e.g. biological testing. The possibility of hidden variable values is regarded as being a problem. In specific cases, the standard deviation in relation to the average value should be stated (content uniformity measurement). Averaging is not permitted when checking mixtures. Averaged results from multiple injections of a sample solution are accepted as 1 result. If results for multiple determinations are obtained that are inside and outside the specifications, the OOS is to be clarified even if the average value is inside the specifications. Outlier tests |
The application of outlier tests is subject to restrictions and their application is to be defined in the testing procedure beforehand. In general, an "outlier" must be regarded more as an indication of sample inhomogeneity and not of variability in the analysis. Outlier tests are permissible for biological testing with high variability. According to USP, an "outlier" should only be declared invalid in exceptional cases. For validated methods with little variability and homogeneous samples, an outlier test is only intended for statistical review of the results and therefore may not be used to declare invalidity. They are not permissible for content uniformity und dissolution analyses. |
Regarding laboratory investigations (see figure 14.H-10): There is no reason why a different analyst should carry out the retesting (Renger, 1999) (PhRMA, 1998). It is always possible/expedient to specify the number of determinations (e.g. in the SOP) in advance. In fact, investigations often reveal new aspects that must be taken into consideration. In response to the accusation that analysis is carried out until the appropriate result is obtained (FDA), American companies prefer to refer to this as the gathering of additional results (PhRMA, 1998). In the case of averaging, it is asserted that variable measured values must always be obtained for each method. It is not feasible to permit averaging for higher variability (biological tests) and not for lower variability. Due to the method precision, it is proposed that the quantity of determinations needs to be established and not the average according to the type of test (Renger, 1999) (PhRMA, 1998). In addition, only the average and not the individual values should be assessed with reference to the specification. Detailed statistical considerations have been put forward to explain why the requirement in the guideline is counterproductive. As a basic principle therefore, averaging is to be permitted. Note: The FDA indicated that they were prepared to compromise on this issue. More attention should be paid to the use of outlier tests (Renger, 1999) (PhRMA, 1998) as they can in fact be used to exclude values that clearly deviate. Additionally, the option of determining the batch assay using the individual values of the Content Uniformity testing providing all of these fulfil the Content Uniformity criteria (Renger, 1999) has also been requested.
| 5. Concluding the investigation |
|---|
The batch quality is assessed based on the investigation results and the approval decision is made. |
A. Interpretation of investigation results If a cause of the OOS result is found that leads to the result being declared invalid, it should not be used for the batch assessment. If the OOS result demonstrates that the product quality is impaired (OOS confirmed), the release decision must take this into account. A confirmed OOS result must lead to the batch being rejected and destroyed. An appropriate decision must be taken with respect to inconclusive results. Statistical approaches (outlier tests) are generally not permissible for chemical analyses. |
B. Reporting For registration batches, the authorities must be informed within 3 working days about an OOS result (FAR, Field Alert Report). |
An OOS result alone should not lead to rejection and destruction of the batch - instead, the points listed above should be taken into consideration (PhRMA, 1998). More attention should be paid to batch assessment based on risk analyses. In addition, a glossary with clear and binding definitions is required (Renger, 1999). Finally, Völler, the pharmaceutical director, has requested for problem-oriented troubleshooting to be compulsory in production where one OOS result is confirmed (Renger, 1999). It should be noted that implementation in practise has been repeatedly and continuously checked and declared suitable by the FDA. If the procedure for OOS investigations were to be restricted purely to the example given in the guideline, this would be difficult to comprehend particularly in view of the fact that self-responsibility is being referred to at every turn.
14.H.4 Example for handling of an OOS result
The implementation of the FDA Guideline is shown by means of an example. Many other options are undoubtedly possible (Häusler, 1999).
The procedure must be illustrated by a flow diagram (see figure 14.H-13). In practise, the use of forms to document the individual steps has proven extremely useful (see figure 14.H-14, figure 14.H-15 and figure 14.H-16).
Example:
Following the content HPLC analysis of product® batch 272 MFD 1299, it is discovered that one of the two values is outside the specification. According to the company SOP, an OOS investigation must be carried out to determine whether the "true" value is inside or outside. Note: all solutions must be retained until the investigations have been concluded.
As the first step, the supervisor must be informed (cf. flow diagram). It must be clarified together with the supervisor whether this is an apparent analytical error . To this end, a formal check is carried out using the Investigation of OOS results (report stage 1) form (see figure 14.H-14). The sample is identified in the header (label) and the reason for the investigation is stated in the section entitled OOS result. The clarifications are documented with the help of the checklist (YES/NO). If a statement does not apply (no injection for recording an IR spectrum), a cross is inserted at N/A (not applicable). If no apparent analytical error can be found based on these data, this must be entered at diagnosis and a cross inserted for the initiation of OOS investigation stage 2. Corresponding entries may be made at measures. The form must be dated and signed by the analyst and supervisor.
Before further investigations are carried out during the next step, the subsequent procedure must be written down in the so-called testing protocol (see figure 14.H-15). This must show:
| Tasks in testing protocol |
|---|
|
A justification for the procedure must be given and the form must be signed (and therefore approved) by the analyst and supervisor prior to implementation.
These specifications are then meticulously implemented and the results evaluated. The results are summarised in a report in the form entitled Investigation of OOS results (report stage 2) (see figure 14.H-16). It is recommended that all results are entered. Conclusions must then be drawn and it must be stated which individual values will be entered in the result (information on certificate). In our example, both original values will be declared invalid and not taken into account in the result. The error category of the original OOS result must also be recorded. The processing of the investigation is also extremely important. At measures, a statement must be made explaining how OOS results are to be avoided in future. The testing procedure which is clearly not "state-of-the-art" must also be updated.
Note: The procedure shown is based on a SOP that requires an OOS investigation for cases like this. In light of the versions (Renger, 1999) (PhRMA, 1998) of the FDA Draft Guidance by representatives of the industry, averaging can also be described as being permissible in the SOP and consequently, the result can be evaluated as being within the specifications in this particular example.
|
|
|
|
|
14.H.5 Trend tracking
According to the FDA, clear and complete records of OOS occurrences must be kept (in paper or electronic form). One of the follow-up activities of an OOS result is periodic review of the cases concerned. Fur the purposes of assessment, an OOS result may not be regarded in isolation, instead, the system should be considered as a whole. Trending makes it possible for potential risks to be revealed at an early stage therefore assisting in the prevention of future OOS results.
The product, the equipment, the methods and also the analysts are subject to trending. If the assay values for one product are frequently outside the specification, this may be a sign that a previously undetected manufacturing problem exists. Equipment that has a tendency to produce OOS results may have to be repaired, maintained or recalibrated. The possibility of increasing the frequency at which calibration and maintenance is carried out or replacement of equipment in extreme cases should be considered. An analogous procedure applies for the methods. As the example shows, a method may possibly have to be formulated in more detail. Finally, the staff must also be included in the trend investigation. If (OSS) errors are frequently caused by a particular member of staff, this person may have to receive additional more in-depth training.
Necessary measures are to be derived from OOS results. A priority must be to make corrections by applying suitable measures. Spurious influencing factors must be eliminated to prevent unnecessary additional OOS results in future. This will also have a direct impact on cost savings. The most urgent measures have already been referred to during the trend analysis.
Summary The handling of OOS results is the central issue of every inspection by the authorities. Companies understand the need to comply with and implement the GMP Guidelines in this area. The release analyses of active pharmaceutical ingredients and drug products are included in the scope where specifications have been defined in the submission file for marketing authorisation, in pharmacopoeias or by manufacturers. As a basic principle, the investigation of OOS results must be carried out according to a protocol. A flow diagram and forms make it easier for all parties concerned to comply with their own OOS SOP. The investigations must be processed carefully, efficiently, impartially and must also be fully documented and be based on scientific facts. Preventative measures must be taken based on the conclusions drawn from the investigation to avoid further OOS results. OSS results should be prevented through regular monitoring of trends (product, equipment, method, analysts). Severe measures will be taken as a consequence of non-compliance with the requirements of the authorities (FDA). Offences will be regarded as (attempted) fraud. Apart a refusal to grant approval for the USA market, this may also lead to further legal action and shutdown of business operations (in USA). There are still considerable deficiencies in the draft guideline that require clarification and specification by the FDA. |
