Robust metrics for assessing the performance of different verbal autopsy cause assignment methods in validation studies

Murray, Christopher JL; Lozano, Rafael; Flaxman, Abraham D; Vahdatpour, Alireza; Lopez, Alan D

doi:10.1186/1478-7954-9-28

Population Health Metrics

Table 2 Range of values for selected cause-specific and overall metrics of individual cause assignment and CSMF estimation for two different hypothetical VA assignment methods across 500 test datasets where the cause composition of the test datasets has been randomly varied.

From: Robust metrics for assessing the performance of different verbal autopsy cause assignment methods in validation studies

	Method 1				Method 2
Cause A	Mean	Median	Max	Min	Mean	Median	Max	Min
Sensitivity	0.70	0.70	0.70	0.70	0.80	0.80	0.80	0.80
Specificity	0.95	0.95	0.96	0.94	0.95	0.95	0.96	0.94
Absolute CSMF error	0.08	0.06	0.29	0.00	0.05	0.04	0.19	0.00
Relative CSMF error	0.74	0.24	53.38	0.00	0.71	0.15	53.48	0.00
Chance-corrected concordance	0.55	0.55	0.55	0.55	0.70	0.70	0.70	0.70
Estimated versus true regression	Intercept	Slope		RMSE	Intercept	Slope		RMSE
	0.52	0.64		0.00	0.52	0.74		.00
Cause B	Mean	Median	Max	Min	Mean	Median	Max	Min
Sensitivity	0.60	0.60	0.60	0.60	0.60	0.60	0.60	0.60
Specificity	0.69	0.69	0.97	0.42	0.70	0.70	0.98	0.42
Absolute CSMF error	0.17	0.15	0.57	0.00	0.17	0.15	0.57	0.00
Relative CSMF error	4.50	0.37	229.07	0.00	4.43	0.37	228.56	0.00
Chance-corrected concordance	0.40	0.40	0.40	0.40	0.40	0.40	0.40	0.40
Estimated versus true regression	Intercept	Slope		RMSE	Intercept	Slope		RMSE
	0.30	0.29		0.11	0.30	0.30		0.11
Cause C	Mean	Median	Max	Min	Mean	Median	Max	Min
Sensitivity	0.35	0.35	0.35	0.35	0.35	0.35	0.35	0.35
Specificity	0.69	0.69	0.73	0.64	0.73	0.73	0.82	0.64
Absolute CSMF error	0.20	0.19	0.63	0.00	0.19	0.17	0.63	0.00
Relative CSMF error	6.75	0.50	793.85	0.00	6.01	0.49	780.54	0.00
Chance-corrected concordance	0.03	0.03	0.03	0.02	0.03	0.03	0.03	0.02
Estimated versus true regression	Intercept	Slope		RMSE	Intercept	Slope		RMSE
	0.31	0.03		0.01	0.26	0.08		0.03
Overall causes	Mean	Median	Max	Min	Mean	Median	Max	Min
Kappa	0.26	0.28	0.47	0.00	0.30	0.33	0.53	0.00
Total absolute CSMF error	0.46	0.45	1.26	0.01	0.42	0.37	1.26	0.03
CSMF accuracy	0.75	0.75	1.00	0.37	0.77	0.80	0.98	0.36

Back to article page

ISSN: 1478-7954

Contact us

General enquiries: journalsubmissions@springernature.com