Patients frequently present to emergency departments (EDs) with headache. About 4.5% of total ED attendance in the United States is attributable to headache,1 1% to 6% of which is caused by non-traumatic subarachnoid haemorrhage (SAH).2 3 4 Among the volume of neurologically intact patients with severe acute headache, identifying the 10% with ‘walking SAH’—patients with SAH but maximum Glasgow Coma Scale score and normal neurological examination—is particularly difficult.5 Specifically, those patients have good Hunt and Hess grading and generally better prognosis.6 Failure to identify those SAH patients would jeopardise those patients’ otherwise good outcomes. A case series demonstrated that 25% of aneurysmal patients with SAH were misdiagnosed during their initial medical evaluations, 38% of which had clinical grade 1 or 2 at the time of misdiagnosis.7 Overall, 24% of patients deteriorated before the correct diagnosis was made, with poor or worse final outcomes. Most of the misdiagnoses of SAH cases were caused by failure to perform computed tomography (CT) imaging. Another study reported that 12% of patients with SAH were initially misdiagnosed, of which 19% had normal mental status at first contact, and these misdiagnoses were associated with worse quality of life at 3 months and increased risk of death or severe disability at 12 months.4 Again, failure to conduct CT scanning was the most common cause, accounting for 73% of diagnostic errors. However, conducting a CT scan on every single patient who attends an ED for headache may not be practical, in consideration of radiation exposure to patients and resource implications. For such purposes, clinical prediction rules including the Ottawa SAH rule have been developed for identification of low-risk patients who can be discharged safely without a CT scan or other imaging (Table 1).4 However, those clinical prediction rules have not been well validated in the Asian Chinese population. The primary objective of the current study is to validate the Ottawa SAH rule in the Asian Chinese population. The secondary objective is to identify possible modifications to improve its accuracy.

This was a retrospective cohort study conducted in the EDs of two regional hospitals in Hong Kong. With daily attendance of approximately 600 and 350 patients, respectively, Tuen Mun Hospital and Pok Oi Hospital together serve a population of over 1 million. All patients are coded according to principal diagnosis following the International Classification of Diseases, Ninth Edition (ICD-9).8 Case recruitment had two phases. First, we searched the hospital’s electronic database for ICD-9 codes indicating headache symptoms, related syndromes, and diseases that may present with a primary complaint of headache (online Supplementary Appendix). In the second phase of case inclusion, the medical records of all cases retrieved in the first phase were screened for eligibility to be included in the current study according to the inclusion and exclusion criteria.

All patients aged ≥16 years who presented with acute headache to the study centres from July 2013 to June 2016 were included. Acute headache was defined as non-traumatic headache that reached maximal intensity within 1 hour, with an interval of <14 days from headache onset to presentation. The clinical details of each retrieved case were screened for inclusion eligibility by both written and electronic medical records. Cases with obvious pathology (eg, frontal sinusitis) were excluded. Exclusion criteria included age <16 years, history of trauma within the last 7 days (collapse associated with headache onset leading to head injury was not an exclusion), history of previous SAH, known cerebral aneurysm or cerebral neoplasm, >14 days since symptom onset, altered mental state, Glasgow Coma Scale score <15 on presentation, and new focal neurological signs. Patients were still included if they had recurrent ED attendance during the study period.

A detailed manual review of written and electronic medical and radiological records was conducted to obtain the following data for eligible cases: duration and quality of headache, presence of thunderclap headache, neck pain, limited range of neck movement, loss of consciousness (LOC), onset with exertion, arrival by ambulance, failure of ambulation in previously ambulatory patients, associated symptoms of dizziness and vomiting, history of benign headache syndrome, concomitant anticoagulant usage or known bleeding diathesis, presenting Glasgow Coma Scale score, blood pressure and heart rate (the first recorded values in the ED), neurological deficits, imaging, lumbar puncture, definitive diagnosis, and neurological outcome. Criteria not documented in the medical records were presumed to be absent. Standardised data collection forms were deployed for data entry by a single investigator.

Subarachnoid haemorrhage was defined in accordance with Perry et al9: subarachnoid blood on CT scan, xanthochromia in cerebrospinal fluid, or red blood cells in the final tube of cerebrospinal fluid, with positive angiography findings (ie, an aneurysm or arteriovenous malformation on cerebral angiography). All CT films were reviewed by both an experienced emergency physician and a radiology fellow, with outcome decided by consensus.

The study outcomes included the sensitivity of the Ottawa SAH rule (Table 1) and the impact on the accuracy of the Ottawa SAH rule of addition of the following clinical predictors to the proposed clinical decision rule: systolic blood pressure (SBP) >160 mm Hg, diastolic blood pressure >100 mm Hg, vomiting, failure of ambulation in previously ambulatory patients, bleeding diathesis or on anticoagulants, and existence of a benign headache disorder that could account for the headache.

R 3.4.1 for Windows (R Foundation for Statistical Computing, Vienna, Austria) was employed for analysis, and a 5% significance level was adopted. Continuous data were presented as mean and standard deviation if normally distributed. Categorical variables were shown as frequencies and percentages. For univariate analysis, comparison was performed between patients in the non-SAH and SAH groups using independent samples t tests, Chi squared tests, and Fisher’s exact test where appropriate. Predictors that were significant in the univariate analysis were entered into the logistic regression model by a forward stepwise method based on likelihood ratios. Adjusted odds ratios (AORs) and P values were calculated for each predictor. The Hosmer-Lemeshow goodness-of-fit test was adopted for model calibration. Model discrimination was evaluated by the area under the receiver operating characteristic (ROC) curve of the predicted probabilities. Collinearity was explored with variance inflation factors. Net reclassification improvement and integrated discrimination improvement indices were calculated to assess the improvement of model prediction with the addition of significant variables to the Ottawa SAH rule.

The Ottawa SAH rule (Table 1) was applied to calculate the sensitivity, specificity, positive and negative predictive values, positive and negative likelihood ratios, and their corresponding 95% confidence intervals (95% CIs). The modified Ottawa SAH rule was created on the basis of the additional independent predictors included in the logistic model and the diagnostic characteristics evaluated.

Sample size was calculated to yield 80% power at a 5% significance level. These calculations assumed SAH prevalence of 6.5%, and the original derivation paper of the Ottawa SAH rule achieved 100% sensitivity and 15.3% specificity.4 With two-tailed hypothesis testing, to achieve the same specificity and 10% variation of sensitivity, a sample of 500 subjects with 31 cases of SAH would be required. Sample size calculation was performed with NCSS PASS 11 (Version 11.0.10).

A total of 1816 potential headache cases during the study period were retrieved from the hospitals’ databases. After a detailed review of clinical information, 500 eligible cases were included in the analysis, in 50 of which SAH was confirmed (10%) [Fig 1]. There were missing values for major components of the Ottawa SAH rule in 16% of the included cases. Two out of the 50 SAH cases had negative CT results (Fisher scale 1), and both of these cases were detected by xanthochromia in cerebrospinal fluid extracted by lumbar puncture. In terms of angiographic findings, angiography was not performed in five patients; six patients had normal angiograms; two had arteriovenous fistulae; one had moyamoya disease; two had cerebral amyloid vasculopathy; and the SAHs of the remaining 34 (68%) patients were aneurysm-related.

The demographic and clinical characteristics of the included cohort are shown in Table 2. The SAH group was contrasted with the non-SAH group in terms of various clinical characteristics. Lumbar puncture was performed in 50 patients, and two patients had xanthochromia in the extracted cerebrospinal fluid. Another 48 patients had SAH diagnosed by CT. Logistic regression (Table 3) revealed that LOC (AOR=16.3; P<0.001) and thunderclap headache (AOR=12.4; P<0.001) were the strongest predictive parameters for SAH. In addition to the parameters in the Ottawa SAH rule, vomiting and SBP >160 mm Hg were demonstrated to be independent predictors of SAH. The Hosmer-Lemeshow goodness-of-fit test demonstrated satisfactory model calibration (P=0.986). Area under the ROC curve for the Ottawa SAH rule was 0.819 (95% CI=0.782-0.851) [Fig 2]. The variance inflation factors of predictors ranged from 1.03 to 1.15, indicating non-significant multicollinearity. The modified Ottawa SAH rule was defined as positive prediction of SAH with the occurrence of any of the following criteria: vomiting, SBP >160 mm Hg, or any of the parameters of the Ottawa SAH rule (Table 1). The area under the ROC curve of the modified Ottawa SAH rule in combination with vomiting and SBP >160 mm Hg increased to 0.870 (95% CI=0.837-0.898). Non-parametric comparison of the areas under the ROC curves demonstrated a statistically significant difference (P=0.041). The net reclassification improvement index was 0.158 (95% CI=0.007-0.309; P=0.040), and the integrated discrimination improvement index was 0.622 (95% CI=0.353-0.891; P<0.001). Both indices indicate that the addition of vomiting and SBP >160 mm Hg would improve the model’s discriminatory and predictive capacity.

Table 4 describes the diagnostic characteristics of various clinical prediction rules that predict SAH in patients with acute headache. The Ottawa SAH rule achieved 94% (95% CI=82.5-98.4%) sensitivity and 32.9% (95% CI=28.6-37.5%) specificity. The modified Ottawa SAH rule in combination with both vomiting and SBP >160 mm Hg produced sensitivity of 100% (95% CI=91.1-100%) and specificity of 13.1% (95% CI=10.2-16.7%).

For 349 out of the 500 included patients, positive predictions would have been rendered by the Ottawa SAH rule, as one or more of its criteria were satisfied (69.8%). For 441 out of the 500 patients, positive predictions would have been rendered by satisfying one or more criteria of the modified Ottawa SAH rule (88.2%). The clinical implication is that 69.8% and 88.2% of the included patients would have required CT according to the Ottawa and modified Ottawa SAH rules, respectively (Table 4). In our cohort, CT was performed in the ED on a total of 481 (96.2%) patients. Thus, application of the Ottawa and modified Ottawa SAH rules can reduce CT administration by 26% and 8%, respectively.

Three patients with SAH were not identified by the Ottawa SAH rule. All three of them were relatively young (aged 20-38 years). Two of them were initially discharged home, with initial negative imaging for SAH, and were diagnosed upon re-attending the accident and emergency departments. Of those two, one patient later developed signs of meningeal irritation with xanthochromia revealed by lumbar puncture, and the other patient was called back to the hospital 7 days later after a retrospective CT report found hydrocephalus and collapsed in the medical ward. A repeat CT scan showed diffuse SAH. Those patients’ mild symptoms at first presentation that led to their initial discharges might account for their falsely negative Ottawa SAH rule findings. In addition, patients who wanted to go home might have been more tolerant of pain and more reluctant to describe alarming symptoms. The former patient was initially diagnosed with reversible cerebral vasoconstriction syndrome, and SAH was noted when the patient subsequently re-attended the ED. This might explain his atypical presentation in comparison with other patients with SAH who had ruptured intracranial aneurysms and tended to present with more florid symptoms from the beginning. The latter patient was diagnosed with a ruptured anterior communicating artery aneurysm. The third patient had a positive CT scan, and digital subtraction angiography showed a ruptured distal internal carotid aneurysm. All three patients experienced vomiting, and two of them had SBP >160 mm Hg.

Various investigations have attempted to identify clinical parameters to predict SAH among patients presenting with non-traumatic headache. The identified predictors include: age >40 years, neck pain or stiffness, LOC, onset with exertion, arrival by ambulance, vomiting at least once, diastolic blood pressure >100 mm Hg, and SBP >160 mm Hg.3 10 11 In the last decade, clinical prediction rules have combined various predictors to provide better accuracy in diagnosis of SAH.4 12 13 The Ottawa SAH rule is one of the best known ones, and it has been subject to external validation.4 An external validation study of the Ottawa SAH rule reported sensitivity of 100% and specificity of 7.6%.14

The SAH incidence rate in our study was noticeably higher than that in the original derivation study and other validation studies.11 14 15 This is not consistent with previous epidemiological studies that reported Chinese people to have a lower rate of aneurysmal SAH than that of other populations.16 One possible explanation is that our inclusion criteria were more stringent, as headache cases with obvious accountable causes were excluded. The mean age and SBP of our cohort were similar to those of other studies.12 13 14 15 Only 8.8% of our included patients reported thunderclap headache, much lower than the 78% to 89% reported in previous studies.4 14 However, the causes of headache in patients with non-SAH headache are mostly benign, and the pattern of benign headache syndromes may be different in the Chinese population compared with that in Caucasians (ie, migraine is more prevalent in Caucasians).17 The rate of onset during exertion was much lower in our cohort than in those of previous studies, as were neck pain and reduced range of movement.14 15 The exact reason for the latter is unknown. One possible explanation is that there are more cervicogenic headaches and occipital neuralgia in the other study cohorts. Conversely, we reported higher rates of arrival by ambulance, vomiting, and LOC,14 15 although one study excluded unwitnessed LOC.14 Loss of consciousness, thunderclap headache, vomiting, SBP >160 mm Hg, neck pain or limited neck range of movement, and age >40 years were found to be significant independent predictors of SAH. Onset during exertion had a high AOR, but its statistical significance was limited by its low incidence and limited sample size.

Both the original derivation study and the validation study by Bellolio et al14 reported that the Ottawa SAH rule had 100% sensitivity.15 In contrast, our study found that the Ottawa SAH rule was only 94% sensitive. The sensitivity of our study was limited by the low rates of onset during exertion, thunderclap headache, and neck pain or limited neck range of movement in our included patients. This might be partly attributable to our retrospective design, as unreported criteria were assumed to be absent. Moreover, the term thunderclap headache might be interpreted differently by different patients. Specificity was higher in our study than in others (32.9% in our study compared with 8%-15% in other studies).14 15 This may be attributed to the difference in the clinical characteristics of patients with non-traumatic headache in the Asian Chinese population compared with those in the original derivation study and other validation studies, which were mostly Caucasians.

Adding two independent predictors for SAH (vomiting and SBP >160 mm Hg) to the Ottawa SAH rule to produce a Modified Ottawa SAH rule improved its accuracy in terms of sensitivity. We found that three patients with SAH could not be identified by the Ottawa SAH rule. All three were relatively young, and two of them presented initially with mild symptoms and were discharged after brain CT did not show SAH. One patient was diagnosed with reversible cerebral vasoconstriction syndrome, which might present differently from the more common aneurysmal SAH. In a validation study,12 20% of patients with SAH were CT-negative, and most of them had posterior communicating artery aneurysm or normal digital subtraction angiography. However, our three patients did not share those clinical characteristics. Integration of vomiting and SBP >160 mm Hg to the model detected those cases and further improved sensitivity to 100% in our cohort.

The specificity of the Ottawa SAH rule was demonstrated to be low (15%) in the original derivation study.4 This implies that among patients without SAH, only 15% had negative predictions by the Ottawa rule, while the remaining 85% had positive predictions. The high false-positive prediction rate may have implications in terms of excessive unnecessary CT scans ordered: it may result in unnecessary radiation exposure, and the surge of CT requests might strain ED resources. With higher specificity in our cohort, we found that application of the Ottawa and modified Ottawa SAH rules can reduce CT use by 26% and 8%, respectively. A UK study found that if the Ottawa SAH rule had been applied, the CT investigation rate would have been much higher (59% to 74%) than the actual rate of 37%.18 Another UK study reported a similar CT investigation rate of 61.7% with the application of the Ottawa SAH rule, which was significantly higher than the rate of 54.2% in actual practice.19 A review surmised that while the Ottawa SAH rule seemingly can rule out SAH, in actual practice, it might increase the frequency of CT investigations.11 However, there is still a lack of impact analysis regarding the effects of the Ottawa SAH rule on patients’ neurological outcomes and mortality.

A clinical decision rule was recently proposed by Kimura et al20 in 2016. In their 1561-patient multicentre observational study, the authors aimed to identify concrete, unambiguous predictors for SAH, avoiding subjective terms like ‘thunderclap headache’. The EMERALD (Emergency Medicine, Registry Analysis, Learning and Diagnosis) SAH rule criteria are SBP >150 mm Hg, diastolic blood pressure >90 mm Hg, blood sugar >115 mg/dL (6.9 mmol/L), or serum potassium <3.9 mEq/L (3.9 mmol/L). Hyperglycaemia has been well reported in patients with SAH. Most studies have focused on the prognostic value of blood glucose, but there have been no other reports on the use of glucose levels for assistance with SAH diagnosis in the literature. Similarly, hypokalaemia has been reported in patients with SAH, which was postulated to be related to increased catecholamine secretion after SAH, resulting in higher intracellular potassium uptake and reduced serum potassium levels. While it requires blood sampling, the EMERALD SAH rule has been reported to have 100% sensitivity and 14.5% specificity, the latter of which is higher than that of the Ottawa SAH rule (8.8% in the study). Thus, more unnecessary CT scans could be avoided with the implementation of simple bedside point of care testing. However, the biological plausibility and external validity of this study might be affected by the lack of evidence about the mechanism of hyperglycaemia and hypokalaemia in patients with SAH, and further, patients with known diabetes mellitus were not excluded from this study. Patients with known cerebral aneurysm or new focal neurological deficits were also not excluded. Because CT scans would almost certainly be ordered for those patients, it might restrict this rule’s usefulness for detection of ‘walking SAH’. We cannot evaluate this rule, as our cohort was unlikely to undergo glucose and potassium sampling in the ED, and so far, we have not found any external validation studies for this clinical decision rule. Nevertheless, it is worth exploring whether the addition of blood glucose and potassium levels to the Ottawa SAH rule could improve its specificity and reduce unnecessary CT administration.

This study has several limitations. First, its retrospective design is prone to information bias. In total, 16% of the predictors were missing values in this study. With missing values, the prevalence of the predictors may be reduced, with potential effects on their diagnostic accuracy. Second, tracing of the outcome of whether or not the patients had SAH was limited because of the study’s retrospective nature. Further, if patients with SAH did not re-attend public hospitals but received treatment in private hospitals, the outcomes may have been missed. Third, as one study centre contains a neurosurgical department, there is a risk of referral bias. Cases diagnosed in other hospitals and referred to the study centre were excluded. In addition, as both study centres are located in the same cluster, SAH cases diagnosed at one study centre are often transferred to the other study centre for neurosurgical consultation at the ED there. Great care was taken to crosscheck between patients at the two study centres to avoid duplicate entry.

Although we exhaustively searched for all eligible cases, there was still a chance of selection bias, as some cases that were eligible according to the inclusion criteria may have been missed. While the Ottawa SAH rule is very sensitive, it is only applicable to a very specific group of patients with headache. Patients with headache that took marginally >1 hour to peak would be excluded. This greatly limits the rule’s clinical applicability throughout the population: one validation study reported that only 9% of patients with headache in an ED were applicable.14 The modified Ottawa SAH rule lacks an external cohort for validation in this study, and validation with an independent multicentre prospective cohort would be required to establish external validity.

In conclusion, the Ottawa SAH rule demonstrated high sensitivity. Addition of vomiting and SBP >160 mm Hg to the Ottawa SAH rule as criteria may increase its sensitivity.

All authors made substantial contributions to the concept or design of the study, acquisition of data, analysis or interpretation of data, drafting of the article, and critical revision for important intellectual content.

All authors have disclosed no conflicts of interest. All authors had full access to the data, contributed to the study, approved the final version for publication, and take responsibility for its accuracy and integrity.

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Approvals from the Hospital Authority New Territories West Cluster Ethics Committee were obtained.

1. Perry JJ, Stiell IG, Wells GA, et al. Attitudes and judgment of emergency physicians in the management of patients with acute headache. Acad Emerg Med 2005;12:33-7.

2. Vermeulen M, van Gijn J. The diagnosis of subarachnoid haemorrhage. J Neurol Neurosurg Psychiatry 1990;53:365-72. Crossref

3. Perry JJ, Stiell IG, Sivilotti ML, et al. High risk clinical characteristics for subarachnoid haemorrhage in patients with acute headache: prospective cohort study. BMJ 2010;341:c5204. Crossref

4. Perry JJ, Stiell IG, Sivilotti ML, et al. Clinical decision rules to rule out subarachnoid hemorrhage for acute headache. JAMA 2013;310:1248-55. Crossref

5. Edlow JA, Malek AM, Ogilvy CS. Aneurysmal subarachnoid hemorrhage: update for emergency physicians. J Emerg Med 2008;34:237-51. Crossref

6. Hunt WE, Hess RM. Surgical risk as related to time of intervention in the repair of intracranial aneurysms. J Neurosurg 1968;28:14-20. Crossref

7. Mayer PL, Awad IA, Todor R, et al. Misdiagnosis of symptomatic cerebral aneurysm. Prevalence and correlation with outcome at four institutions. Stroke 1996;27:1558-63. Crossref

8. International Classification of Diseases. Ninth Edition. Geneva, Switzerland: World Health Organization; 1977.

9. Perry JJ, Sivilotti ML, Sutherland J, et al. Validation of the Ottawa Subarachnoid Hemorrhage Rule in patients with acute headache. CMAJ 2017;189:E1379-85. Crossref

10. Lui CT, Tsui KL, Kam CW. Nuchal pain predicts subarachnoid haemorrhage in severe headache patients. Hong Kong J Emerg Med 2008;15:212-7. Crossref

11. Carpenter CR, Hussain AM, Ward MJ, et al. Spontaneous subarachnoid hemorrhage: a systematic review and meta-analysis describing the diagnostic accuracy of history, physical examination, imaging, and lumbar puncture with an exploration of test thresholds. Acad Emerg Med 2016;23:963-1003.Crossref

12. Mark DG, Hung YY, Offerman SR, et al. Nontraumatic subarachnoid hemorrhage in the setting of negative cranial computed tomography results: external validation of a clinical and imaging prediction rule. Ann Emerg Med 2013;62:1-10. Crossref

13. Kelly AM, Klim S, Edward S, Millar N. Sensitivity of proposed clinical decision rules for subarachnoid haemorrhage: an external validation study. Emerg Med Australas 2014;26:556-60. Crossref

14. Bellolio MF, Hess EP, Gilani WI, et al. External validation of the Ottawa subarachnoid hemorrhage clinical decision rule in patients with acute headache. Am J Emerg Med 2015;33:244-9. Crossref

15. Kowalski RG, Claassen J, Kreiter KT, et al. Initial misdiagnosis and outcome after subarachnoid hemorrhage. JAMA 2004;291:866-9.Crossref

16. de Rooij NK, Linn FH, van der Plas JA, Algra A, Rinkel GJ. Incidence of subarachnoid haemorrhage: a systematic review with emphasis on region, age, gender and time trends. J Neurol Neurosurg Psychiatry 2007;78:1365-72. Crossref

17. Stewart WF, Lipton RB, Liberman J. Variation in migraine prevalence by race. Neurology 1996;47:52-9. Crossref

18. Matloob SA, Roach J, Marcus HJ, O’Neill K, Nair R. Evaluation of the impact of the Canadian subarachnoid hemorrhage clinical decision rules on British practice. Br J Neurosurg 2013;27:603-6. Crossref

19. Yiangou A, Nikolenko N, Noreikaite J, Thondam S. Impact of subarachnoid haemorrhage Canadian clinical decision rule for investigation of acute headache, a retrospective case note review. Lancet 2017;389:S103. Crossref

20. Kimura A, Kobayashi K, Yamaguchi H, et al. New clinical decision rule to exclude subarachnoid hemorrhage for acute headache: a prospective multicenter observational study. BMJ Open 2016;6:e010999. Crossref