Software used to interpret results of electrocardiograms (ECGs) can vary significantly in the detection of arrhythmia and ability to detect critical results and should not represent the sole criterion for clinical decision making, according to a study published in the Journal of Electrocardiology.
The investigators sought to compare the performance of 7 different mainstream ECG programs for the interpretation of a large set of ECGs. Programs were assessed for their ability to detect abnormal rhythms and accurately flag acute coronary syndrome for priority review.
The investigators reviewed 2123 ECG records obtained from consecutive adult and pediatric patients from hospitals and databases across Europe, Australia, and the United States. Digital ECGs were converted into analog format, which were then replayed 3 times each on each electrocardiograph and classified by the manufacturers’ interpretation programs. To render the statements independent of the manufacturers’ specific wording, the investigators created categories for the analysis of arrhythmic events, including “sinus rhythm,” “atrial fibrillation/flutter,” and “other arrhythmias.” For the analysis of acute coronary syndromes, the investigators included statements that indicated “critical values” or “critical test result” indicative of acute coronary syndrome. In cases where the interpretation statements of all 7 programs did not agree, the ECG results (without interpretation) were sent to experienced cardiologists for review. The rates of false-positive and false-negative results were used to assess program performance.
Of 2123 ECGs included in the rhythm analysis, the programs agreed for 77.9% of cases indicating sinus rhythm, 6.5% indicating atrial fibrillation/flutter, and 0.1% indicating abnormal rhythms. After interpreting and including cases in which 1 or more programs did not agree with the others, 88.6% of ECGs were indicative of sinus rhythm, 8.6% of atrial fibrillation/flutter, and 2.8% of abnormal rhythms. False-positive rates ranged from 2.1% to 5.5% for non-sinus rhythm, from 0.7% to 4.4% for atrial fibrillation/flutter, and from 1.5% to 3.0% for other abnormal rhythms. False-negative rates ranged from 30.5% to 55.9% for abnormal rhythms. For both false-positive and false-negative rates, the programs with the lowest and highest rates differed significantly from the other programs. In acute coronary syndrome analysis, programs with the highest frequency of flagging cases for review were more than double the programs with the lowest flagging frequency. The reviewers were more likely to flag cases than the programs, and some flagged cases twice as often as others. Between the program and majority reviewer interpretations, agreement was reached in 46% to 62% of cases. Overall, false-positive rates for flagging acute coronary syndrome were low, with the highest rate being 1.6%. False-negative rates were high (>50%).
Study limitations include the need to convert digital ECG samples into analog recordings, the possibility that discrepancies in amplification, signal processing, and filtering may affect the results, and the fact that cardiologists did not review ECGs for which no critical indication was given.
“Critical test results or acute infarction statements should not be used as the sole criterion for priority processing,” noted the study authors.
De Bie J, Martignani C, Massaro G, Diemberger I. Performance of seven ECG interpretation programs in identifying arrhythmia and acute cardiovascular syndrome. J Electrocardiol. 2019;58:143-149.