Did SignalPET's AI “match the best radiologist”?

Our status

Partially supported. Equivalence was not demonstrated.

The narrower observation is supported: in this dataset and under this reference-standard method, the reported accuracy point estimates were close—90.2% for SignalRAY and 91.0% for the best-accuracy radiologist. The study tested for a difference and reported p = 0.0561.

The stronger wording “Equivalent Accuracy” is not demonstrated by the study design. Failure to detect a significant difference does not itself establish equivalence, non-inferiority, or equal clinical diagnostic utility.

The evidence

A small reader study of coded descriptive findings.

The supporting paper was published in Frontiers in Veterinary Science on 21 February 2025. It compared SignalRAY with 11 board-certified veterinary radiologists holding ECVDI or ACVR credentials.

• 50 radiographic studies: 40 canine and 10 feline, retrospectively drawn from one institutional PACS.
• Reader coverage: each radiologist reported 24 or 25 studies; the AI processed all 50. Each study received a mean of 5.46 radiologist reports.
• AI task: images only, producing coded normal/abnormal descriptive findings. The AI did not receive the clinical context supplied to radiologists.
• Human task: full written descriptive and diagnostic reports, later manually reduced to codes matching the AI output.
• Differential diagnoses: excluded from evaluation because the tested AI did not generate them.
• Software timing: SignalRAY was accessed in July 2022. The paper says it was continuously updated and had no version numbers.

What the data show

Close overall accuracy, different error profiles.

“Best” and “median” depend on the metric. The same radiologist was not necessarily best for accuracy, sensitivity and specificity.

The headline comparison

SignalRAY's accuracy estimate was 0.8 percentage points below the best-accuracy radiologist: 90.2% versus 91.0%. The paper's Bonferroni-corrected proportion z-test reported p = 0.0561 for that difference. Against the median-accuracy radiologist, SignalRAY was 2.7 points higher; the paper reported p = 0.0022.

The error profile matters

SignalRAY was less sensitive to abnormal findings than nine radiologists, with the paper reporting p < 0.0001 for those comparisons. It was more specific for normal findings than ten radiologists, also reported at p < 0.0001. In plain language: this system was stronger at agreeing that a finding was normal than at finding abnormalities.

The paper did not report confidence intervals for the headline performance estimates or the AI–best-radiologist difference.

Claim escalation analysis

Where the wording becomes stronger.

1. 1 · Data90.2% versus 91.0% overall accuracy.

   SignalRAY also had lower sensitivity and higher specificity than the best-accuracy radiologist.

2. 2 · Statistical resultNo significant accuracy difference detected by the study's z-test.

   The reported p-value was 0.0561. This is a failure to detect a difference, not a positive equivalence result.

3. 3 · Authors' interpretation“Almost as well as the best” for descriptive findings.

   The authors also wrote that the AI was “no less accurate,” while acknowledging lower abnormality sensitivity and no differential diagnoses.

4. 4 · Vendor interpretationThe study “validates” expert-level interpretation.

   SignalPET's research article presents the result as confirmation of reliable, specialist-comparable performance.

5. 5 · Marketing claim“Equivalent Accuracy” and “matched the best radiologist.”

   This wording converts an inconclusive difference test into an affirmative equivalence message.

Where the claim holds

The point estimates were close in this study.

• SignalRAY's 90.2% overall accuracy was close to the best radiologist's 91.0% within the study's constructed consensus framework.
• The AI's overall accuracy exceeded the median-accuracy radiologist's estimate under the reported analysis.
• The AI had high specificity for normal findings and performed particularly well in the study's low-ambiguity group.
• The authors' narrower formulation—almost as well for coded descriptive findings—is more consistent with the data than a broad claim of equivalent radiologist-level interpretation.

Where it stretches

“Matched” carries more certainty and scope than the evidence.

Limitations

Promising comparison, narrow evidentiary base.

• Small, single-source dataset. Fifty studies from one institutional PACS cannot represent the full case mix, acquisition quality and disease spectrum of general practice.
• Class imbalance. Consensus labeled 83.9% of 16,434 findings normal and 16.1% abnormal. Calling every finding normal would yield about 84% accuracy, making sensitivity essential context.
• Non-independent reference standard. Observer consensus included SignalRAY, the system being evaluated.
• Coding assumptions. An unmentioned finding was treated as normal, and insignificant abnormalities in radiologist reports were coded abnormal. These decisions may affect human–AI comparisons.
• Reader and case clustering. The reported proportion z-tests did not model repeated findings within cases and repeated readings by radiologists.
• No equivalence framework. There was no prespecified margin, equivalence confidence interval or power calculation for proving a clinically acceptable difference.
• Evaluation-set independence unclear. The paper describes train, validation and test splits used in model development but does not establish whether these 50 evaluation studies were absent from every development dataset.
• Different exposure and tasks. The AI assessed all 50 studies using images alone; each radiologist assessed roughly half and received signalment, history and clinical signs.
• Version traceability. The continuously updated July 2022 system had no version number, limiting reproducibility and relevance to later releases.

Vendor involvement

SignalPET funded the study and supplied operational support.

The later peer-reviewed commentary reported no funding and no commercial or financial conflicts.