Abstract

This report describes a new approach to categorizing ocular injury using Military Health System data, the application of an algorithm to a dataset, and the verification of the results using an audit of clinical data. Based on health care encounter data, an algorithm was developed to systematically document the occurrence of specific complications and medical procedures in the 12 months following initial ocular injuries. The injuries were classified into 1 of 2 groups: "uncomplicated injury" with no complications or medical procedures and "complicated injury" with complications and/or medical procedures. Injuries in the latter group were further classified by severity into low, moderate, and high strata based on a ranking of complications and medical procedures. From 2016 through 2019, 12,664 complicated ocular injuries and 49,016 uncomplicated injuries were identified among active duty U.S. military members. The vast majority (84%) of complications were concurrent or occurred within 30 days following the injury. The 3 most common complications (orbital floor fracture, iridocyclitis and recurrent corneal erosion) accounted for 52% of complications. These findings underscore the importance of accurate classification of complex ocular injuries to inform studies in multiple areas including injury prevention, the development of clinical guidelines, and health services research.

What Are the New Findings?

The Vision Center of Excellence (VCE) developed a classification algorithm that uses longitudinal data (including diagnoses of complications and medical procedures related to the initial ocular injury) to more accurately classify ocular injuries sustained by U.S. service members. The results of the algorithm's classification were verified through medical record review.

What is the Impact on Readiness and Force Health Protection?

Ocular injuries present an ongoing threat to readiness and retention of service members. Robust surveillance methods are needed to account for non-specific and often inconsistent diagnosis coding in electronic health records. The increased accuracy of classification of ocular injuries provides critical data to inform prevention and treatment initiatives.

Background

Ocular injuries have the potential to negatively impact the readiness and retention of U.S. service members and incur a significant cost to the Military Health System (MHS).¹ The availability of large administrative/billing data sources provides the opportunity to surveil the burden and trend of ocular injuries. It is a challenge, however, to categorize ocular injuries based on electronic health record (EHR) data because coded diagnoses may be non-specific and inconsistent in resource-constrained settings, such as an emergency room or a primary care clinic. Ocular injuries frequently involve more than 1 anatomic structure, and have been described in multiple publications as complex injuries.^2–4 Surveillance of ocular injuries in the U.S. Armed Forces is conducted by the Tri-service Vision Conservation and Readiness Program of the Army Public Health Center and the Armed Forces Health Surveillance Division which identifies cases based on initial diagnoses in administrative health records.⁵

The Vision Center of Excellence (VCE) was established by congressional mandate in 2008 as a center of excellence in the prevention, diagnosis, mitigation, treatment, and rehabilitation of military eye injuries.⁶ The VCE developed the military ocular injury case definition in support of multiple initiatives, including improved utilization of the Defense and Veterans Eye Injury and Vision Registry (DVEIVR). This registry contains information to track diagnoses, interventions/treatments, and follow-up for each case of significant eye injury sustained by a member of the U.S. Armed Forces while serving on active duty. The DVEIVR permits accurate ocular injury burden reporting and estimation of health care burden across the MHS. To improve the accuracy of classification of ocular injuries in the MHS, the VCE developed a novel approach to characterizing ocular injury that uses information on multiple diagnoses and procedures from MHS administrative data. This approach allows classification of ocular injuries by their severity and can be used for multiple types of initiatives including monitoring trends and targeting significant ocular injuries in the U.S. military members. The objective of this study was to provide accurate information on the type and severity of ocular injuries to inform strategic planning for medical readiness, resource allocation, and prevention programs.

Methods

Ocular injury data were obtained by applying the VCE case definition for ocular injury to the Military Health System Management and Analysis Reporting Tool (M2), a longitudinal administrative data warehouse that contains electronic medical records of hospitalization and ambulatory medical encounters in both military medical treatment facilities (direct care) and civilian facilities (purchased care). Additional data on ocular injuries documented in deployed settings were obtained from the Theater Medical Data Store (TMDS). Health care records of both active component and activated guard/reserve service members were captured based on beneficiary category in the M2 and included in these analyses. Guard and reserve service members receive the same care entitlements as those in the active component if they were on active duty for more than 30 days, or may qualify for Line of Duty Care if injured while on active duty for fewer than 30 days.

The classification algorithm (Figure) developed by the VCE used multiple criteria including 320 International Classification of Diseases, 10th Revision (ICD-10) diagnosis codes for ocular injuries (Table 1), 332 ICD-10 diagnosis codes for potential complications (Table 2), and 1,543 Procedural Classification System (PCS) and ICD Current Procedural Terminology (CPT) codes for outpatient procedures associated with ocular injuries (Table 3) to assess and classify the severity of the injuries.

Of note, the complications include conditions with potential impacts on readiness and retention, such as corneal ulcers, retinal breaks, and orbital fractures. The included list of procedures was based on the results of the ophthalmology knowledge, skills, and abilities (KSA) workgroup. This ad hoc workgroup, consisting of military ophthalmologists with deployment experience, reviewed outpatient ophthalmic procedures and developed a tiered system to determine readiness to deploy based on the number of procedures performed in the military treatment facilities. The VCE mapped CPT codes to ICD-10 PCS codes to ensure capture of procedures conducted in inpatient settings. As a result, individuals with a KSA tier of 1, 2, or 3 were classified by the algorithm as complicated injury if not already identified as such based on complications (Figure).

All encounter data in the M2 and the TMDS that included diagnoses of ocular injuries or complications for active duty members were collected from 2016 through 2020. Data were line-based, and each line represented 1 visit (direct care) or 1 claim (purchased care). Data from different care locations were consolidated into 1 comprehensive dataset to accommodate all variables and visit dates were used to assess temporality of encounters. The algorithm was then applied to identify the presence of the pre-specified injuries, complications, and medical procedures for each individual during the 12 months following the initial injury date. Those injured individuals without complications and no medical procedures corresponding to KSA tier 1 through 3 were classified as having suffered an uncomplicated injury; otherwise, individuals were classified as having a complicated injury. For those individuals with uncomplicated injury, if 60 days passed after the date of the injury without a subsequent health care encounter for an uncomplicated ocular injury, then an encounter for an uncomplicated injury after the 60-day period would be counted as a newly incident uncomplicated injury. For complicated injuries, the severity level was further categorized as low, moderate, or high based on the highest rank of presented complications (Table 2) or prescribed medical procedures (Table 3). Of note, complicated injuries were not subject to an incidence rule like that for uncomplicated injuries; an individual was assumed to sustain only 1 complicated injury during the study period. Complicated injuries with complications only, medical procedures only, or both were counted. For complications, the number of different complication types for each individual was determined; for individuals with more than 1 type of complication, all complications were included (Figure). Based on the incidence rule, individuals may suffer from multiple uncomplicated injuries but only one complicated injury was counted per individual during the study period. Data analysis and case classification were conducted using R, version 4.0.3 (2020, R Core Team).

The results of the classification by the algorithm were verified by the VCE team of medical auditors using clinical data. Study auditors had clinical backgrounds in eye care and were trained on ocular care medical record abstraction and classification of ocular injuries using DVEIVR business rules for data abstraction. Cohen's kappa was used to measure agreement between the outcomes of the VCE algorithmic classification and record review. A total of 1,000 individuals (500 complicated and 500 uncomplicated) with relevant encounter data from 2018 were randomly selected for medical record validation. These data were used to develop and troubleshoot the algorithm. For example, ICD-10 codes S05.70X*, S05.71X, and S05.72X were added to the complication list after this condition (avulsion of eye) was found documented in the medical record following initial injury documentation. The VCE is currently evaluating outcome data of available cases for complicated injury using information in the DVEIVR to further explore the utility of injury severity stratification.

Results

From 2016 through 2019, 12,664 complicated and 49,016 uncomplicated injuries were identified among 59,653 active duty service members using the classification algorithm (Figure). Of the individuals with complicated injuries, 6,119 had complications only, 5,554 had medical procedures only, and 991 had documentation of both complications and medical procedures. Individuals with complicated injuries were further classified as having injuries of low severity (n=1,864), moderate severity (n=8,785), and high severity (n=2,015). A total of 2,834 individuals with uncomplicated injuries had multiple occurrences of such injuries.

Approximately half (52%) of complications were concurrent (diagnosed on the same day as the ocular injury) or immediately following the initial injuries. Eighty five percent and 91% of complications occurred within 30 and 90 days of the documentation of the initial injury, respectively. Of the individuals (n=7,089) with complicated injuries, 1,184 had more than one complication.

Among the types of complications (n=12,365), the top 3 complications included orbital floor fracture (20%), iridocyclitis (19%), and recurrent corneal erosion (13%). The top 10 complications accounted for 85% of total complications (Table 4).

Of the 1,000 individuals selected for verification of the algorithm, 844 records were available for chart review using the Armed Forces Health Longitudinal Technology Application (AHLTA) or the Joint Legacy Viewer. Of the 445 cases classified as complicated by the algorithm, 443 (99.6%) were found to be correctly categorized based on medical record review. Of the 399 cases classified as uncomplicated by the algorithm, 392 (98.2%) were found to be correctly categorized based on record review. Agreement of classifications between the algorithm and chart review was high (kappa=0.96).

Editorial Comment

Through the application of the described algorithm which assesses multiple diagnoses and/or prescribed surgical procedures, ocular injuries were categorized and stratified into groups that more accurately reflect the complexity and severity of these injuries to improve classification of ocular injury. The resulting categorizations were verified using an audit of clinical data with a high degree of agreement. The clinical reality of the complex nature of ocular injuries is especially difficult to capture by conventional surveillance which relies on a single diagnostic code in administrative records. Categorizing using a single code may produce inaccurate data that are difficult to interpret from a public health, clinical, or operational perspective. The implications for accurate case definitions by systematical evaluation of associated complications and procedures for surveillance and research are significant.

Furthermore, the results of this analysis show that 85% of complicated injuries had the first documented complication within the first 30 days of the initial injury. This approach can be adapted to surveillance systems which require update at an interval less than 12 months; for example, a quarterly update can capture and categorize the vast majority (90%) of cases. The potential miscategorization due to limited length-of-care episodes is likely minimal due to the frequency of concurrent complications and procedures. Annual analysis would enable further evaluation of all associated complications and procedures, allowing for refinement of public health interventions targeting primary prevention, detailed outcome analysis to inform tertiary prevention efforts, and quantitation of health care burden. In addition, identification of cases for review in conjunction with detailed clinical information available in the DVEIVR could identify best practices in clinical treatment of military ocular injuries.

An ocular injury frequently involves multiple anatomic structures with multiple complications. For example, contusion might be associated with hyphema (an accumulation of blood in the anterior chamber of the eye), vitreous hemorrhage, orbital floor fracture, traumatic cataract, and optic nerve injury which cannot be captured by conventional surveillance methods relying on first-recorded ICD code. Data from the application of the algorithm showed that 15.6% of complicated injuries had multiple complications (ranging from 2 to 6 unique complications).

There are limitations to this approach. It is known that EHR data are mainly for administrative uses and have no guarantee for accuracy of coded diagnoses for surveillance and research.^7,8 The limitations of administrative/claims data for secondary use have been described previously, specifically in regards to ophthalmic care. For example, it has been noted that patients may be misclassified due to misdiagnosis, miscoding, as well as coding practices of multiple providers with varying coding experience.⁹ Furthermore ocular injuries can be complex and involve multiple anatomic regions which cannot be appropriately captured by the conventional surveillance approaches. It is necessary to utilize multiple medical data points, e.g., diagnoses, prescribed procedures and medications, to increase accuracy of case identification based on electronic health record.¹⁰

In addition, ocular burns and corrosion were not included in this analysis; these injuries will need to be explored separately with modifications to the case definition with special consideration given to expected complications more likely in these injuries (for example, ocular surface complications such as dry eye syndrome or exposure keratopathy, and lid complications such as entropion or ectropion).

It was assumed that the new case definition captured all ocular injuries without considering the possibility of missing true cases of ocular injury in M2. It is, however, extremely unlikely that there were ocular injuries which leave no trace of clinical information to be captured by this case definition. This study did not select samples of uncaptured cases as negative controls for validation because it would have needed a large sample to identify false negatives given the extremely rare likelihood. Therefore, no estimation was made of sensitivity, specificity, positive predictive value, and negative predictive value for the case definition.

The main purpose of injury surveillance efforts is to inform the development, prioritization, and execution of injury prevention initiatives. The results of this study demonstrated that categorization of ocular injuries could be obtained by evaluation of episodes of care recorded in administrative records for the secondary uses of surveillance and research. The longitudinal approach ensures more reliable characterization of injuries which offers several advantages: 1) it removes reliance on single ICD-10 codes for categorization of ocular injuries, 2) it increases the sensitivity and specificity of the case definition of ocular injury, 3) it provides a measure of severity of ocular injuries, and 4) it provides increased granularity of data for ocular injuries categorized as complicated, with capture of multiple associated complications and procedures. The resultant outcome provides a rich, clinically-relevant dataset for outcome and health system analysis.

Author affiliations: Department of Defense/Veterans Affairs Vision Center of Excellence, Defense Health Agency Research and Development Directorate (COL Reynolds, Dr. Gu).

Disclaimer: The contents, views, or opinions expressed in this publication are those of the author(s) and do not necessarily reflect the official policy or position of the Defense Health Agency, Department of Defense, or the U.S. Government.

Disclosure: Utilization of data and development of the case definition methodology was completed under the DHA Office of Research Protection #935913.

References

1. Frick KD, Singman EL. Cost of military eye injury and vision impairment related to traumatic brain injury: 2001-2017. Mil Med. 2019;184(5–6):e338–e343.
2. Colyer MH, Mazzoli RA. Complex ocular trauma outcomes and system capabilities: lessons from a combat zone and implications for national eye trauma care. Eye (Lond). 2021;35(8):2069-2070.
3. Gonnering RS. Is oculofacial surgery complex or merely complicated? Curr Opin Ophthalmol. 2018;29(5):434–439.
4. Harvey MM, Justin GA, Brooks DI, Ryan DS, Weichel ED, Colyer MH. Ocular trauma in Operation Iraqi Freedom and Operation Enduring Freedom from 2001 to 2011: A Bayesian network analysis. Ophthalmic Epidemiol. 2021;28(4):312–321.
5. Hilber DJ. Eye injuries, active component, U.S. Armed Forces, 2000–2010. MSMR. 2011;18(5):2–7.
6. National Defense Authorization Act for Fiscal Year 2008, Public Law 110–181, section 1623. 2008.
7. Horsky J, Drucker EA, Ramelson HZ. Accuracy and completeness of clinical coding using ICD-10 for ambulatory visits. AMIA Annu Symp Proc. 2017;2017:912–920.
8. Nicholson A, Tate AR, Koeling R, Cassell JA. What does validation of cases in electronic record databases mean? The potential contribution of free text. Pharmacoepidemiol Drug Saf. 2011;20(3):321–324.
9. Stein JD, Lum F, Lee PP, Rich WL, 3rd, Coleman AL. Use of health care claims data to study patients with ophthalmologic conditions. Ophthalmology. 2014;121(5):1134-1141.
10. Herrett E, Thomas SL, Schoonen WM, Smeeth L, Hall AJ. Validation and validity of diagnoses in the General Practice Research Database: A systematic review. Br J Clin Pharmacol. 2010;69(1):4–14.