Impact of FDA Core Courses on Texas Manufactured Food Inspector Written Observations
Davonna W. Koebrick, LMSW, RS
Sanitarian II
Texas Department of State Health Services
International Food Protection Training Institute (IFPTI)
2012 Fellow in Applied Science, Law, and Policy: Fellowship in Food Protection
Abstract
Texas Department of State Health Services (TDSHS) Manufactured Food Inspectors attended Food and Drug Administration (FDA) Core Courses FD150: Food Good Manufacturing Practices, FD151: Food Inspection Techniques and Evidence Development, and FD152: Food Processing and Technology in groups beginning September 24, 2009 through April 27, 2012 as part of TDSHS Manufactured Food Regulatory Program (MFRP) Standard Two: Training Program. This study assessed whether there was a difference in the reported critical and noncritical observations documented in TDSHS routine inspection reports after FDA training as compared to inspection reports prior to FDA training. The results of the study were inconclusive given the inability to control for all of the possible variables that could have contributed to any variances. Despite the outcome, three challenges to evaluating and comparing written observations were identified as needing to be considered by regulatory programs implementing the MFRP Standards, particularly Standard 4: Inspection Audit Program. This standard requires the State program to conduct quality assurance reviews to evaluate the effectiveness of the inspections program and recognize trends in inspectional coverage (U.S. Food and Drug Administration, 2010). The challenges included lack of detailed written observations, lack of written guidance to determine the appropriate regulation(s) the observation violates, and lack of written guidance to determine when an observation is critical versus noncritical.
Background
The Texas Department of State Health Services (TDSHS) began implementing the Manufactured Food Regulatory Program Standards (MFRP Standards) in 2009 as a requirement of the Food and Drug Administration (FDA) Food Protection Rapid Response Team (RRT) and Program Infrastructure Improvement Prototype Project. The MFRP Standards were developed by a committee of FDA and state regulatory officials in response to a June 2000 U. S. Office of Inspector General (OIG) report that made several recommendations to FDA to address shortcomings identified in FDA’s oversight of state food firm inspections. The report highlighted the need for equivalency among Federal and State food safety standards, inspection programs, and enforcement practices (U. S. Office of Inspector General, 2000). The MFRP Standards Committee identified ten areas crucial to a high-quality regulatory program charged with protecting the public from foodborne illness, including Regulatory Foundation, Training Program, Inspection Program, Inspection Audit Program, Food-related Illness and Outbreaks and Response, Compliance and Enforcement Program, Industry and Community Relations, Program Resources, Program Assessment, and Laboratory Support (U. S. Food and Drug Administration, 2010).
The MFRP Standards for Training Program, Inspection Program and Inspection Audit Program help ensure that food protection agencies have competent inspectors and consistent inspections. The Training Program standard defines coursework and field training at the basic and advanced level for a person conducting food inspections. The Inspection Program standard describes how an effective program, through written policy and procedures, requires an inspector to recognize significant violative conditions or practices, record findings, and distinguish between significant and insignificant observations, and isolated incidents versus trends. The Audit Program standard describes the basic quality assurance reviews necessary to: (1) evaluate the effectiveness of the inspection program, (2) recognize trends in inspectional coverage, and (3) identify best practices used to achieve quality inspections and sample collections. This standard uses Field Inspection Audits, Inspection Report Audits and Sample Report Audits to assess the effectiveness of the inspection program. In particular, the Field Inspection Audit and Inspection Report Audit evaluate inspectors against criteria such as the ability to recognize significant violative conditions or practices, and whether inspectors are able to distinguish between significant and insignificant violative conditions (U.S. Food and Drug Administration, 2010).
TDSHS selected FDA’s Level 1 Manufactured Foods Curriculum to meet the Training Program standard. The curriculum includes a combination of online and three face-to-face core courses: Food Good Manufacturing Practices (FD150), Food Inspection Techniques and Evidence Development (FD151), and Food Processing and Technology (FD152). TDSHS inspectors that had not completed FD150, FD151 and FD152 began attending the courses in large groups in September 2010 and most had completed the courses by April 30, 2012.
The TDSHS Manufactured Foods program inspects wholesale and retail food manufacturers operating in Texas. Retail food establishments that manufacture food are inspected under 25 Texas Administrative Code (TAC) Chapter 229, Good Manufacturing Practice and Good Warehousing Practice in Manufacturing, Packing, or Holding Human Food that closely follows, with state-specific modifications, 21 Code of Federal Regulations (CFR) Part 110 Current Good Manufacturing Practice (GMP) in Manufacturing, Packing, or Holding Human Food. Violative conditions observed by inspectors during an inspection are called “observations” and are recorded on Form E-14 issued to the food manufacturer at the conclusion of the inspection.
Problem Statement
The impact of FDA Food Core courses FD150, FD151 and FD152 on TDSHS inspector reporting of significant violative conditions or practices during routine GMP inspections of food manufacturers in Texas is unknown.
Research Question
Did attending FDA Food Core courses FD150, FD151 and FD152 have an impact on inspector reporting of significant violative observations during routine GMP inspections of Texas food manufacturers?
Methodology
To assess whether completion of the FDA courses impacted reported written observations, and because there was no control group available, inspection reports from before and after course completion were selected for review. In order to be included in the study, the inspection report: had to be conducted during the same time periods in 2010 and 2012; had to be a routine GMP inspection of a food manufacturer that processes food; the inspector must have completed all three FDA courses during the same time period; and the inspector must have conducted inspections prior to and after the courses. Additionally, the ratio of retail manufacturers to wholesale manufacturers was kept the same for both sets of reports reviewed.
The last FDA course completed by groups of inspectors was FD151 on April 27, 2012. Post-course inspection reports were collected from May 1, 2012 through July 31, 2012 due to availability and the onset of the IFPTI Cohort III Fellowship in July 2012. To control for seasonal variability, pre-course reports were collected from the same time period of May 1st through July 31st in 2010 prior to inspectors attending the FDA courses which began in September 2010. In addition to the course completion requirements, only inspectors that conducted manufactured food inspections prior to and after completing the courses were included.
Inspection reports selected were limited to routine GMP inspections of food manufacturers because FD150, FD151 and FD152 cover basic GMP inspections. FDA provides other courses that cover specialized processing such as acidified foods and low acid canned foods. Although companies that conduct specialized processing must also comply with GMPs, this study was conducted to assess the impact of the courses on basic food manufacturing operations. To ensure that all categories under the GMPs and product labeling were applicable for review during the inspection, focused inspections such as sampling, recalls, and some field investigations were excluded from the study, as well as private label manufacturers and warehouses that do not process food.
The 2010 inspection list generated by the TDSHS Regulatory Programs Regulatory Automation System (RAS) Portal database resulted in 127 inspections that fit the study parameters involving 41 wholesale manufacturers and 86 retail manufacturers. The 2012 inspection reports were randomly selected until 41 wholesale manufacturers and 86 retail manufacturers were identified that fit the study parameters. Reports for routine inspections resulting in regulatory action such as warning letters were only available for review in person while reports for routine inspections that did not result in regulatory action were available remotely on the TDSHS server. If a “review in person” report was selected, that report place was maintained. If the “review in person” report was unavailable, the report was replaced with another “review in person” report that resulted in regulatory action.
Inspection report written observations were entered into a 2010 Microsoft Excel spreadsheet with columns for the inspection report number, inspection date, inspector number, the food product code, food risk, study observation classification, 2009 Food Code observation classification, Texas law citation, and the Food Code citation. Pre-defined cell drop boxes were used to utilize the spreadsheet filtering features and to assist with observation categorization consistency.
Observations were categorized as follows: personnel, plant and grounds, sanitary operations, sanitary facilities and controls, equipment and utensils, production and process controls, labeling, unable to determine, and other. The “other” category included licensing and non-applicable observations such as expired over-the-counter drugs. The “unable to determine” category was used for observations that were not written clearly.
Within each category, the observation was subcategorized as “critical” or “noncritical” using 2009 Food Code risk designations Priority, Priority Foundation and Core; FDA’s FD320 State Food Contract Auditing manual (U.S. Food and Drug Administration, 2011); and International Food Protection Training Institute (IFPTI) Fellowship “Evaluating Violations” exercise (8/15/2012) as guidance. In the “Labeling” category, undeclared allergens or no manufacturer name and location was designated as “critical”. Similar reported observations in an inspection report that were numbered separately on the E-14 were grouped into one observation in the spreadsheet. For example, if five separate foods with the same labeling violation were numbered 1 through 5 on the E-14, they were entered as one observation in the spreadsheet. For consistency, unclean food contact surface observations were categorized in the Production and Process Controls category even though they can also be categorized under Sanitary Operations [21 CFR 110.35(d)] in the GMP regulations.
The frequency rates of all 2010 and 2012 reported observation categories and sub-categories were calculated. The frequency rates from 2010 and 2012 were then compared for each category and sub-category using chi-square distribution with one degree of freedom (α = 0.05, X2 = 3.84) (Lipschutz & Lipson, 2011) (Robson, Shannon, Goldenhar, & Hale, 2001).
Results
Inspection reports from 24 TDSHS manufactured food inspectors met the study parameters. Of the 127 inspection reports from 2010 that met the study criteria, nineteen did not list any critical or noncritical observations on the E-14. Ten of the 127 inspections from 2010 resulted in warning letters. Of the 127 randomly selected reports for 2012, thirty-eight inspections did not list any critical or noncritical observations on the E-14, and twelve of the remaining reports resulted in warning letters. The increase from 19 inspection reports without observations in 2010 to 38 reports without observations in 2012 was a statistically significant change (X2 = 6.33).
Excluding the inspection reports without observations, a total of 449 reported observations were entered into the spreadsheet for 2010 from the remaining 108 reports. A total of 425 reported observations were entered for 2012 from the remaining 89 reports.
Overall the percentage of total critical observations and noncritical observations in all categories did not change significantly between the 2010 and 2012 inspections, with 34.30% critical in 2010 and 34.59% in 2012 [See Section A of Table 1]. Likewise, the “unable to determine” category did not show a significant change. The ratio of total critical to noncritical observations was 0.586 and 0.588 respectively for 2010 and 2012 also indicating no significant change [See Section F of Table 1].
When total critical and noncritical observations are organized by the six GMP and Labeling categories [See Section B of Table 1], there is a significant change in the Sanitary Facilities and Controls and the Production and Process Controls categories. The total in Sanitary Facilities and Controls decreased from 66 observations in 2010 to 35 in 2012 (x2 = 7.895). The two areas within this category with the largest decreases were plumbing, from 19 total observations to 11, and handwashing facilities, from 32 total observations to 15. The total observations in Production Processes and Controls increased from 66 to 86 (x2 = 3.848). The area of raw material and ingredient handling increased by 19 observations and maintenance of equipment, utensils and finished food containers in acceptable condition increased by 13 observations.
There were no significant changes when the percentage of critical observations to total observations within each category was compared [See Section C of Table 1]. However, when the percentage of critical observations per total observations in the year [See Section D of Table 1] was reviewed for each category, there was a significant change in Sanitary Facilities and Controls, with 10.2% in 2010 decreasing to 5.9% in 2012 (x2 = 5.115).
When the ratio of critical to noncritical observations within each category was compared [See Section E of Table 1], only Equipment and Utensils changed significantly from a ratio of 0.448 in 2010 to 0.190. In 2010, for every one critical observation there were 2.2 noncritical observations, and in 2012, for every critical observation there were 5.3 noncritical observations. The largest increase within the category was in the construction and maintenance of equipment and utensils, from 26 total observations (84.6% noncritical) to 40 total observations (90.0% noncritical).
Conclusions
There are too many uncontrolled variables in the study to conclude that the FDA courses did or did not impact TDSHS food manufacturer inspector reporting of critical violations. However, there were findings within the data that warrant additional exploration. First, the fact that there was no change between the ratio of critical to noncritical observations between the 2010 reports and the 2012 reports could indicate that TDSHS food manufacturer inspectors view all observations as equally valid because the observation is out of compliance with a specific regulation, as opposed to viewing the level of risk that an observation could result in the adulteration of the manufactured food. Second, out of 254 inspection reports and 874 observations reviewed, none of the observations were for failing to wash hands even though there was a total of 47 pre- and post-training observations regarding the adequacy of facility handwashing sinks. Third, there was a reduction in the number of observations regarding unshielded lights from 19 in 2010 to 10 in 2012. However, the written observations still did not adequately describe how food was exposed under the unshielded lights even though this regulation [21 CFR 110.20(b)(5)] was covered in the FD150 course along with the other GMP regulations.
Three challenges were encountered in this study that may also have a direct impact on an inspection program’s ability to evaluate training and inspection program effectiveness and to identify trends in inspectional coverage as required under the MFRP Audit Standard. The first challenge was lack of detailed written observations. Some observations reviewed were classified as noncritical because of a lack of information e.g., equipment observed to be unclean without describing whether the unclean area was a food-contact surface. An observation could have been further strengthened if a description of the food being processed was included, particularly if the food was ready-to-eat and/or potentially hazardous. The FD151 course included a section on how to write a good observation.
A second challenge was determining the appropriate regulation for the observed violative condition. While some observations are easy to match with the applicable regulation, some violative conditions appear to fall under more than one rule e.g., unclean food contact surfaces fall under Sanitary Operations [21 CFR 110.35(d)] and Production and Process Controls [21 CFR 110.80(b)(1)].
The third challenge was determining whether a violative condition is critical or noncritical. While the FDA 2009 Food Code for retail food establishments provides guidance as to whether a rule is a priority or priority foundation issue based on CDC’s causes of food-borne illness, comparable risk-based guidance is not available for food manufacturing establishments.
Recommendations
Additional inspector guidance and training is needed to enable TDSHS inspectors to consistently compose written E-14 observations that are clear, complete and applicable and to consistently identify and distinguish high-risk critical violations from low-risk observations.
Moreover, inspectors need to receive timely, regular feedback and clarifications on actual written observations from inspections. The Inspection Report audit component of the MFRP Audit standard may need to include a detailed review of written observations in such a way that feedback can be provided to inspectors where specific rules may need to be clarified or reminders given regarding the components of a well-written and actionable observation.
In-depth analysis of category data collected in this study can be used to identify areas where clarification may be needed as to the applicable regulation and the correct interpretation of the regulation. The 2009 Food Code priority, priority foundation and core food-borne illness contributing factor risk levels, which were used during this study to classify reported observations of food manufacturing establishments, could be used as a starting point to develop a guidance document for inspectors to use during inspections, and for field and report auditors to use when evaluating individual inspector performance.
Regulatory programs implementing the MFRP Standards should consider assessing training effectiveness to ensure that selected courses impart the knowledge and job skills against which manufactured food inspector performance is measured. A valid and reliable evaluation tool outside of the accompanying end of course assessment may need to be developed to ensure the classroom course meets the needs of a food inspection program.
Acknowledgments
I would like to sincerely thank all of my colleagues in the Texas Department of State Health Services (TDSHS) Environmental and Consumer Safety Inspections Unit- Foods and the Policy, Standards and Quality Assurance (PSQA) Foods Group for everything you all do on a daily basis to protect the public. Thank you PSQA for pulling all those inspection reports for me and answering all my questions about the work you do. I would especially like to thank Frank Borden, TDSHS Food and Drug Branch Manager, and my supervisor, David Sueltenfuss, South Group Manager, for your support in my participation in the International Food Protection Training Institute (IFPTI) Fellowship. A special thanks to my mentor, Jim Sevchik, and Dan Sowards, my temporary mentor, for your invaluable patience, support and feedback on this project. To all of the IFPTI staff, IFPTI Fellowship subject matter experts, and IFPTI fellows, it has been an honor to be in your company, and I feel blessed to have had the opportunity to share this time with you and learn from you. Most of all, thank you to my husband, Paul, and daughter, Bailey, who always believe in me and are my number one fans.
References
U. S. Food and Drug Administration. (September 2011). State Food Contract Auditing
FD320. Kansas City, Missouri.
U. S. Food and Drug Administration. (September 2010). Manufactured Food
Regulatory Program Standards (MFRPS). Retrieved from
http://www.fda.gov/ForFederalStateandLocalOfficials/PartnershipsContracts/Overview/default.htm
Lipschutz, S., & Lipson, M. (2011). Probability. New York: McGraw-Hill Companies.
Robson, L. S., Shannon, H. S., Goldenhar, L. M., & Hale, A. R. (2001). Guide to
Evaluating the Effectiveness of Strategies for Preventing Work Injuries: How to Show Whether a Safety Intervention Really Works. Cincinnati: National Institute for Occupational Safety and Health.
U. S. Office of Inspector General. (2000). FDA Oversight of State Food Firm
Inspections: A Call for Greater Accountability OEI-01-98-00400.