Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Developing a symptoms-based risk score for infectious syphilis among men who have sex with men

Developing a symptoms-based risk score for infectious syphilis among men who have sex with men Original research Developing a symptoms- based risk score for infectious syphilis among men who have sex with men 1 1 1 Silvia Achia Nieuwenburg , Elske Hoornenborg , Udi Davidovich, 1,2,3 1,2,4 Henry John Christiaan de Vries , Maarten Schim van der Loeff ► Additional supplemental ABSTRACT WHAT IS ALREADY KNOWN ON THIS TOPIC material is published online Background Syphilis incidence is rising among men only. To view, please visit the ⇒ A symptoms- based risk score for infectious who have sex with men (MSM). An online tool based on journal online (http:// dx. doi. syphilis motivating men who have sex with men a risk score identifying men with higher risk of infectious org/ 10. 1136/ sextrans- 2022- (MSM) to seek care has not previously been 055550). syphilis could motivate MSM to seek care. We aimed developed. therefore to develop a symptoms- based risk score for Department of Infectious infectious syphilis. Diseases, Public Health Service WHAT THIS STUDY ADDS Methods We included data from all consultations by Amsterdam, Amsterdam, The ⇒ We developed a risk score based on symptoms Netherlands MSM attending the Amsterdam Centre for Sexual Health and notification for syphilis. Individual Amsterdam Institute for in 2018–2019. Infectious syphilis (ie, primary, secondary Infection and Immunity (AII), symptoms included in the risk score were or early latent syphilis) was diagnosed according to Amsterdam UMC location strongly associated with infectious syphilis, the centre’s routine protocol. Associations between University of Amsterdam, but the risk score had poor sensitivity and symptoms and infectious syphilis were expressed as odds Amsterdam, The Netherlands specificity. Department of Dermatology, ratios (OR), with 95% confidence intervals (CI). Based Amsterdam UMC location on multivariable logistic regression models, we created HOW THIS STUDY MIGHT AFFECT RESEARCH, University of Amsterdam, risk scores, combining various symptoms. We assessed Amsterdam, The Netherlands PRACTICE OR POLICY the area under the curve (AUC) and cut- off based on Department of Internal ⇒ Symptoms- based risk scores for infectious Medicine, Division of Infectious the Youden Index. We estimated which percentage of syphilis cannot be recommended to select MSM Diseases, Amsterdam UMC MSM should be tested based on a positive risk score and location University of for syphilis screening; all MSM at risk of syphilis which percentage of infectious syphilis cases would then Amsterdam, Amsterdam, the should be regularly screened. be missed. Netherlands Results We included 21,646 consultations with 11,594 unique persons. The median age was 34 years Correspondence to Silvia Achia Nieuwenburg, (IQR 27–45), and 14% were HIV positive (93% on in the early latent stage. Long-acting penicillin G, GGD Amsterdam, Amsterdam antiretroviral treatment). We diagnosed 538 cases of an effective and inexpensive antibiotic, remains the 1018 WT, The Netherlands; infectious syphilis. Associations with syphilis symptoms/ snieuwenburg@ ggd. recommended treatment for syphilis. signs were strong and highly significant, for example, OR amsterdam. nl The signs and symptoms of syphilis vary for a painless penile ulcer was 35.0 (CI 24.9 to 49.2) and depending on the stage, and for this reason syphilis Received 15 June 2022 OR for non- itching rash 57.8 (CI 36.8 to 90.9). Yet, none is also known as ‘the great imitator’. This makes it Accepted 21 October 2022 of the individual symptoms or signs had an AUC >0.55. challenging for patients to recognise the disease and The AUC of risk scores combining various symptoms seek a health provider when they are symptomatic. varied from 0.68 to 0.69. For all risk scores using cut- offs The primary stage is characterised by a usually pain- based on Youden Index, syphilis screening would be less ulcer at the site of infection, usually appearing recommended in 6% of MSM, and 59% of infectious within 3 months after acquisition and disappearing syphilis cases would be missed. within 6 weeks. If left untreated, the disease may Conclusion Symptoms- based risk scores for infectious progress to the secondary stage characterised by syphilis perform poorly and cannot be recommended to a non- itching rash. Early latent stage syphilis is select MSM for syphilis screening. All MSM with relevant asymptomatic and can be diagnosed with serolog- sexual exposure should be regularly tested for syphilis. ical tests in patients without a history of primary or secondary symptoms. © Author(s) (or their employer(s)) 2022. Re- use The increasing incidence of syphilis calls for permitted under CC BY- NC. No INTRODUCTION preventive efforts to reduce exposure to the infec- commercial re- use. See rights Syphilis, caused by the bacterium Treponema pall- tion. To reduce risk of ongoing transmission (espe- and permissions. Published idum, remains a public health concern. In many cially in the primary and secondary stage) and of by BMJ. countries, the incidence of syphilis is rising, espe- sequelae, early diagnosis and treatment are key, To cite: Nieuwenburg SA, cially among populations of men who have sex with and it would be helpful if patients would be able to Hoornenborg E, men (MSM). If left untreated, it can lead to serious recognise the disease and be motivated to seek care. Davidovich U, et al. cardiovascular and neurological complications. Therefore, the Centre for Sexual Health of Sex Transm Infect Epub ahead of print: [please include Day Syphilis can also facilitate the transmission of HIV. the Public Health Service of Amsterdam and the Month Year]. doi:10.1136/ It is highly contagious through sexual contact national non- governmental organisation STI- sextrans-2022-055550 during the primary and secondary stage, as well as AIDS Netherlands launched a campaign to create Nieuwenburg SA, et al. Sex Transm Infect 2022;0:1–6. doi:10.1136/sextrans-2022-055550 1 Original research Table 1 Characteristics of all MSM at first consultation in study period; Amsterdam Centre for Sexual Health, July 2018–July 2019 Total Infectious syphilis No infectious syphilis Variables (n=11,594) (n=274) (n=11,320) P value Age in years (median, IQR) 34 (27–45) 36 (29–47) 34 (27–45) 0.009 Country of birth, n (%) Netherlands 7,332 (63.2) 155 (56.6) 7,177 (63.4) 0.020 Othe4,r 4,262 (36.8) 119 (43.4) 4,143 (36.6) Sexual behaviour, n (%) MSM 10,237 (88.3) 260 (94.9) 9,977 (88.1) 0.001 MSMW 1,357 (11.7) 14 (5.1) 1,343 (11.9) HIV status, n (%) Negative 10,005 (86.3) 152 (55.5) 9,853(87.0) <0.001 Positive 1,586 (13.7) 122 (44.5) 1,464 (12.9) Unknown 3 (0.0) 3 (0.0) 0 (0.0) On ART, n (%)* 1,396 (93.0) 103 (89.6) 1,293 (93.3) 0.132 HIV VL as reported by patient, n (%) Undetectable 1,379 (86.9) 105 (86.1) 1,274 (87.0) <0.001 Detectable 33 (2.1) 3 (2.5) 30 (2.0) Notified for syphilis 421 (3.6) 56 (20.4) 365 (3.2) <0.001 STD diagnosis at current visit, n (%) Chlamydia 1,146 (9.9) 52 (19.0) 1,094 (9.7) <0.001 Gonorrhoea 1,290 (11.1) 62 (22.6) 1,228 (10.9) <0.001 Lymphogranuloma venereum 73 (0.6) 8 (2.9) 65 (0.6) <0.001 Hepatitis B 9 (0.1) 1 (0.4) 8 (0.1) 0.084 Hepatitis C 3 (0) 0 (0) 3 (0) 0.788 *For those who are HIV positive, data on ART use of 85 participants were missing. ART, antiretroviral therapy; IQR, interquartile range; MSM, men who have sex with men; MSMW, men who have sex with men and women; STD, sexually transmitted disease; VL, viral load. awareness of syphilis in Amsterdam, the Netherlands. The syphilis. A syphilis diagnosis was made according to the centre’s awareness campaign, which ran from 2018 to 2019, included routine protocol using a combination of clinical signs and symp- an online tool to enable individuals to self- identify symptoms toms and laboratory testing based on the European guidelines that could be related to syphilis, and if indicated motivate them for syphilis management. Serological screening for syphilis to seek care (testing and immediate treatment in case a diagnosis consisted of a treponemal test chemiluminescence immunoassay was made). This online tool included an expert- based algorithm (CLIA, DiaSorin, Saluggia, Italy) confirmed by an immunoblot of various symptoms; we also included a partner notification (INNO- LIA Syphilis, Innogenetics, Ghent, Belgium) and a non- for syphilis in the risk score. The aim of the current study was treponemal test, the rapid plasma reagin (RPR, RPR NOSTICON to assess whether an evidence- based symptoms- based risk score II, bioMérieux, Marcy l’Etoile, France). In persons with suspected could be developed, which could help individuals to self- identify primary syphilis, we additionally performed dark- field micros- possible infectious syphilis. copy of ulcer exudate and a PCR of ulcer scraping. Secondary syphilis was diagnosed if a client had a positive serology with characteristic skin or mucocutaneous lesions. Early latent syph- METHODS ilis was defined as confirmed positive serology without clinical There are few data from the peer-reviewed literature on the asso- manifestations, according to the European guidelines. ciations of symptoms with a current syphilis infection. We there- We described person characteristics by medians and inter- fore developed a syphilis assessment algorithm in consultation quartile ranges (IQRs), and numbers and percentages. Univari- with experts (dermatologists and internist- infectiologist). Based able logistic regression was done using Generalized Estimating on the expert opinions, the following signs and symptoms were Equations (GEE), because of correlated data (multiple visits by included in the risk algorithm: a painful or painless ulcer on the the same person). Logistic regression was done to obtain ORs penis, in the mouth, at the anus or on the skin, with or without with 95% CIs for infectious syphilis diagnosis for each symptom palpable regional lymph nodes, and an itching or non- itching and risk factor. Various multivariable models were constructed rash with or without influenza-like symptoms. In addition, we to enable creation of a series of prediction models and predic- also added partner notification for syphilis. A complete overview tion scores. Our initial risk score A1 was based on the symptoms of the included items is provided in online supplemental table 1. of the expert- based algorithm: ulcer in the mouth, ulcer on the To assess the validity of the expert-based algorithm, we used skin, painless ulcer at the anus with palpable regional lymph data from all consultations by MSM attending the Centre for nodes, painless ulcer on the penis with palpable regional lymph Sexual Health of the Amsterdam Public Health Service, the nodes, painless ulcer at the anus without palpable regional lymph Netherlands, from July 2018 to July 2019, regardless of reason nodes, painless ulcer on the penis without palpable regional of visit. Consultations were at the client’s own initiative and free lymph nodes, painful ulcer at the anus with palpable regional of charge. lymph nodes, painful ulcer on the penis with palpable regional For this study, the outcome of interest was infectious syph- lymph nodes, painful ulcer at the anus without palpable regional ilis, which comprises primary, secondary and early latent 2 Nieuwenburg SA, et al. Sex Transm Infect 2022;0:1–6. doi:10.1136/sextrans-2022-055550 Original research Table 2 Multivariable analysis of possible syphilis symptoms and their association with a diagnosis of infectious syphilis among MSM; Amsterdam Centre for Sexual Health, 2018–2019 Model A* Model B† Model C‡ Coding Symptoms/risk factor aOR (95% CI) βˆ aOR (95% CI) βˆ aOR (95% CI) βˆ 1.m Ulcer in the mouth No Ref Yes 3.2 (0.9 to 12.0) 1.2 1.s Ulcer on the skin§ No Ref Yes 0.7 (0.1 to 4.4) −0.3 1.1.p Painless ulcer on the penis No Ref Yes 36.1 (25.2 to 51.6) 3.6 1.1.1.a Painless ulcer at the anus with lymph nodes¶ No Ref Ref Ref Yes 63.8 (4.1 to 991.8) 4.2 63.6 (4.1 to 989.3) 4.2 65.4 (4.4 to 981.1) 4.2 1.1.1.p Painless ulcer on the penis with lymph nodes No Ref Ref Yes 85.8 (36.6 to 201.0) 4.5 85.7 (36.6 to 200.6) 4.5 1.1.2.a Painless ulcer at the anus without lymph nodes No Ref Yes 1.4 (0.3 to 6.5) 0.4 1.1.2.p Painless ulcer on the penis without lymph nodes No Ref Ref Yes 29.1 (19.5 to 43.4) 3.7 30.0 (20.2 to 44.7) 3.4 1.2.p Painful ulcer on the penis No Ref Yes 21.7 (14.2 to 33.2) 3.1 1.2.1.a Painful ulcer at the anus with lymph nodes No Ref Yes 1.5 (0.2 to 11.7) 0.4 1.2.1.p Painful ulcer on the penis with lymph nodes No Ref Ref Yes 11.5 (4.1 to 32.0) 2.4 13.0 (4.9 to 34.4) 2.6 1.2.2.a Painful ulcer at the anus without lymph nodes No Ref Ref Ref Yes 9.6 (4.9 to 18.6) 2.3 9.6 (5.0 to 18.7) 2.3 9.7 (5.0 to 18.8) 2.3 1.2.2.p Painful ulcer on the penis without lymph nodes No Ref Ref Yes 25.0 (15.6 to 39.9) 3.2 24.7 (15.5 to 39.6) 3.2 2.paso Rash on the palms of the hands or soles of the feet No Ref Yes 2.6 (0.8 to 8.1) 2.1 Itching rash¶ No Ref Ref Ref Yes 7.4 (3.4 to 16.3) 2.0 8.1 (3.8 to 17.2) 2.1 8.0 (3.7 to 17.0) 2.1 2.2Non- itching rash No Ref Ref Ref Yes 54.3 (32.7 to 90.2) 4.0 64.8 (40.4 to 103.7) 4.2 64.8 (40.5 to 103.7) 4.2 3. Notified for syphilis No Ref Ref Ref Yes 4.4 (3.3 to 5.7) 1.5 4.4 (3.3 to 5.7) 1.5 4.3 (3.3 to 5.7) 1.5 β∧ is the regression coefficient of the logistic regression models (i.e. the log of the Odds Ratio). *Model A is based on the symptoms of the expert- based algorithm: ulcer in the mouth, ulcer on the skin, painless ulcer at the anus with palpable regional lymph nodes, painless ulcer on the penis with palpable regional lymph nodes, painless ulcer at the anus without palpable regional lymph nodes, painless ulcer on the penis without palpable regional lymph nodes, painful ulcer at the anus with palpable regional lymph nodes, painful ulcer on the penis with palpable regional lymph nodes, painful ulcer at the anus without palpable regional lymph nodes, painful ulcer on the penis without palpable regional lymph nodes, rash on the palms of the hands or soles of the feet, itching rash, non-itching rash and a partner notification for syphilis. †Model B is based on the symptoms and risk factor (a partner notification for syphilis) that were significantly associated with infectious syphilis: painless ulcer at the anus with palpable regional lymph nodes, painful ulcer at the anus without palpable regional lymph nodes, painless ulcer on the penis with palpable regional lymph nodes, painless ulcer on the penis without palpable regional lymph nodes, painful ulcer on the penis with palpable regional lymph nodes, painful ulcer on the penis without palpable regional lymph nodes, itching rash, non- itching rash and a partner notification for syphilis. ‡Model C is based on model B, but simplified by including painless and painful ulcers (both regardless of lymph nodes): painless ulcer at the anus with palpable regional lymph nodes, painful ulcer at the anus without palpable regional lymph nodes, painless ulcer on the penis, painful ulcer on the penis, itching rash, non-itching rash and a partner notification for syphilis. §Skin is defined as location other than anogenital (anus or penis) or mouth. ¶Rash associated with syphilis: a maculopapular exanthema or erythematous exanthema. aOR, adjusted OR; MSM, men who have sex with men. lymph nodes, painful ulcer on the penis without palpable score. Here we only included the symptoms and risk factor (a regional lymph nodes, rash on the palms of the hands or soles of partner notification for syphilis) that were significantly asso- the feet, itching rash, non- itching rash and a partner notification ciated with infectious syphilis in model A1, without weighing. for syphilis (online supplemental table 1). Risk score B2 included the same factors as risk score B1, but In risk score A1, each factor was awarded 1 point if present with weighing based on the regression coefficients. To optimise and 0 if not, that is, without weighing. Risk score A2 included the risk scores, we also calculated risk scores C1 and C2. Penile the same factors as risk score A1, but here each symptom/factor ulcers were associated with infectious syphilis, independently was given a weight equal to the rounded regression coefficient of lymph nodes being palpable or not. Therefore, risk score of the multivariable analysis. Risk score B1 was a simplified risk C1 included the same factors as risk score B1, but rather than Nieuwenburg SA, et al. Sex Transm Infect 2022;0:1–6. doi:10.1136/sextrans-2022-055550 3 Original research RESULTS In total, 21,646 consultations of 11,594 persons were included in this study (see table 1 which provides details of persons at their first visit in the study period). The median age was 34 years. A total of 1,586 (13.7%) were living with HIV; of the 1,501 with information on viral load, 1,379 (91.9%) reported an undetectable viral load. In total, 421 (3.6%) persons were notified for syphilis. At the 21,646 consultations, 538 cases of infectious syphilis were diagnosed (2.5%). A total of 1270 men (5.9%) reported at least one of the symptoms included in the expert-based risk score. In 260 consultations (1.2%), a penile ulcer was present, in 114 (0.5%) an anal ulcer and in 148 (0.7%) a skin rash. The characteristic rash on hand palms or feet soles was rare, present in 23 (0.1%) consultations. In univariable analysis, all symptoms and the risk factor (a partner notification for syphilis) used were associated with a Figure 1 ROC curve of score C2 among MSM with infectious syphilis; diagnosis of infectious syphilis (online supplemental table 2). Amsterdam Centre for Sexual Health. Risk score C2 is comprised of the The self- reported symptoms: non- itching rash with influenza- following symptoms (between brackets the weights of each symptom): like symptoms (OR 86.8, 95% CI 35.2 to 214.1), rash on the painless anal ulcer with palpable regional lymph nodes (4.2)+painful soles of the feet (OR 78.6, 95% CI 7.1 to 867.6), painless anal anal ulcer with no palpable regional lymph nodes (2.3)+painless penile ulcer with palpable regional lymph nodes (OR 78.6, 95% CI 7.1 ulcer (3.5)+painful penile ulcer (3.1)+itching rash (2.1)+non-itching to 867.6) and painless penile ulcer with palpable regional lymph rash (4.2)+a partner notification for syphilis (1.5). MSM, men who have nodes (OR 76.6, 95% CI 34.0 to 172.6) were most strongly sex with men; ROC, receiver operating characteristics. associated with an infectious syphilis diagnosis. The AUCs of the individual symptoms or signs were between 0.50 and 0.55. In a multivariable analysis including all symptoms (and a including painless and painful ulcers with and without regional partner notification for syphilis) included in the expert-based palpable lymph nodes, we included painless and painful ulcers, risk score (model A), the following symptoms were strongly asso- both regardless of lymph nodes. Risk score C2 included the same ciated with an infectious syphilis diagnosis: painless penile ulcer factors as risk score C1, but with weighing based on the regres- with palpable regional lymph nodes (adjusted OR (aOR) 85.8, sion coefficients. 95% CI 36.6 to 201.0), painless anal ulcer with palpable regional For each risk score, we assessed the area under the curve lymph nodes (aOR 63.8, 95% CI 4.1 to 991.8) and non-itching (AUC). To evaluate the clinical relevance of the risk scores A, rash (aOR 54.3, 95% CI 32.7 to 90.2) (table 2). In model B, B and C, the post- test probability of disease was evaluated. The which is similar to model A, but with non-significant symptoms positive predictive value of having infectious syphilis when omitted, the same symptoms were strongly associated with an having a score above the cut- off was estimated; the cut- off was infectious syphilis diagnosis: painless penile ulcer with palpable based on the Youden Index. The Youden Index measures the effectiveness of a diagnostic marker as follows: sensitivity+spec- regional lymph nodes (aOR 85.7, 95% CI 36.6 to 200.6), non- ificity–1. As there is no generally agreed optimum of sensitivity itching rash (aOR 64.8, 95% CI 40.4 to 103.7) and painless anal and specificity combination, we selected the highest value for ulcer with palpable regional lymph nodes (aOR 63.6, 95% CI the Youden Index as optimal cut-off . Furthermore, we estimated 4.1 to 989.3). Because a painless penile ulcer was strongly asso- which percentage of infectious syphilis cases would be missed, if ciated with infectious syphilis regardless of palpable regional patients would only be tested for syphilis if their score exceeded lymph nodes, and also a painful penile ulcer was strongly asso- the cut-off of the risk score. A p value of <0.05 was consid- ciated with infectious syphilis regardless of palpable regional ered statistically significant. We carried out analyses using Stata lymph nodes, we made model C in which ‘painless penile ulcer’ (V.15.1, StataCorp, College Station, Texas, USA). and ‘painful penile ulcer’ were included rather than the four Table 3 Performance of several risk scores for infectious syphilis among MSM; Amsterdam Centre for Sexual Health, July 2018–July 2019 Infectious syphilis among Infectious syphilis among those with a score of at those with a score below Sensitivity Specificity % to be Risk score*Cut- off† least the cut- offthe cut- off (95% CI) (95% CI) AUC tested Score A1 1.0 220/1,270 318/20,376 40.9 (36.7 to 45.2) 95.0 (94.7 to 95.3) 0.6811 5.9 Score A2 0.9 220/1,226 318/20,420 40.9 (36.7 to 45.2) 95.2 (94.9 to 95.5) 0.6846 5.7 Score B1 1.0 218/1,206 320/20,440 40.5 (36.3 to 44.8) 95.3 (95.0 to 95.6) 0.6811 5.6 Score B2 1.5 218/1,206 320/20,440 40.5 (36.3 to 44.8) 95.3 (95.0 to 95.6) 0.6846 5.6 Score C1 1.0 218/1,206 320/20,440 40.5 (36.3 to 44.8) 95.3 (95.0 to 95.6) 0.6811 5.6 Score C2 1.5 218/1,206 320/20,440 40.5 (36.3 to 44.8) 95.3 (95.0 to 95.6) 0.6846 5.6 *Risk scores are based on the multivariable models as explained in table 2. Risk scores A1, B1 and C1 are based on a summation of all risk factors, each with identical weight (one). Risk scores A2, B2 and C2 are based on a summation of the weights of all risk factors, the weight being identical to the regression coefficient of a factor in the multivariable model (see table 2). †Based on the Youden Index. AUC, area under the curve; MSM, men who have sex with men. 4 Nieuwenburg SA, et al. Sex Transm Infect 2022;0:1–6. doi:10.1136/sextrans-2022-055550 Original research separate items. In this optimised model C, a painless anal ulcer be generalisable to all MSM including those who do not yet seek with inguinal lymph nodes (aOR 65.4, 95% CI 4.4 to 981.1), sexual healthcare. non- itching rash (aOR 64.8, 95% CI 40.5 to 103.7) and painless For the prevention of syphilis, multiple approaches are penile ulcer (aOR 36.1, 95% CI 25.2 to 51.6) were most strongly necessary. An awareness campaign with a symptoms- based risk associated with an infectious syphilis diagnosis. score will likely not prevent the ongoing transmission of infec- Risk score A1, based on the expert-based algorithm, had a tious syphilis. Early identification of the infection and timely sensitivity of 40.9% and a specificity of 95.0%. Based on the treatment are helpful to reduce the duration of infection and cut- off of 1.0 (based on Youden Index), 5.9% of men would onwards sequela. An online symptoms tool might in some cases have to be tested and 59% of syphilis cases would remain unde- be harmful as it may warrant incorrect assessment of no risk. tected. The risk score using weighing (A2) performed similarly. Thus, all MSM who engage in sexual behaviour that places them The other four risk scores (B1, B2, C1, C2) performed almost at higher risk of acquiring STIs should be regularly tested for identical to the risk scores A1 and A2. The AUC was 0.68 for syphilis. Yet, an online tool might help to motivate MSM to seek risk score A1 based on the expert- opinion algorithm (figure 1). help timely in case of suspected symptoms. The AUCs of the other risk scores, including our optimised risk In conclusion, our proposed symptoms- based risk scores score C2, were all also 0.68. Table 3 shows the sensitivity and performed poorly to diagnose infectious syphilis. Thus, we specificity of the various risk scores for infectious syphilis. cannot recommend a symptoms- based risk score to select MSM for syphilis screening. All MSM with relevant sexual exposure should be regularly tested for syphilis. DISCUSSION We aimed to assess whether an evidence-based symptoms- based Handling editor Nigel Field risk score could be developed to identify individuals with infec- Twitter Henry John Christiaan de Vries @henryjdevries tious syphilis. The optimised risk score C2, including six symp- Contributors MSvdL, HJCdV and SN conceptualised and designed the study. SN toms and a partner notification for syphilis, had a sensitivity of performed the data analysis and interpretation of the data. MSvdL reviewed all 40.5% (95% CI 36.3% to 44.8%), a specificity of 95.3% (95% analyses. SN drafted the manuscript. All authors critically revised the manuscript. MSvdL had full responsibility for the work, the conduct of the study, had access to CI 95.0% to 95.6%) and an AUC of 0.68. Based on this score, the data, and controlled the decision to publish. 5.6% of the study population would be eligible for syphilis Funding The authors have not declared a specific grant for this research from any testing and 41% of the cases would be identified. funding agency in the public, commercial or not- for-profit sectors . We had aimed that a symptoms- based risk score would enable Competing interests None declared. MSM to self-identify possible infectious syphilis to increase early testing and diagnosis, and avoid further transmission and compli- Patient consent for publication Not required. cations. To our knowledge, there are no studies on the develop- Ethics approval Ethical approval was not required for this study, as it made use of ment of a symptoms-based risk score for infectious syphilis. One routinely collected data and all data were de- identified prior to analysis. study developed a simple-to- use nomogram to predict the risk Provenance and peer review Not commissioned; externally peer reviewed. of syphilis, and several other studies described the risk factors Data availability statement All data relevant to the study are included in the for syphilis based on sociodemographic, clinical or behavioural article or uploaded as supplemental information. 11–14 data. The main objective of the current study was to develop Supplemental material This content has been supplied by the author(s). a symptoms- based risk score so that individuals would seek help It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not when symptomatic, analogous to similar algorithms for acute have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all HIV infection. Several symptom and behavioural risk scores for 15–18 liability and responsibility arising from any reliance placed on the content. acute HIV have been developed and validated successfully. Where the content includes any translated material, BMJ does not warrant the The symptoms- based risk scores may have the advantage above accuracy and reliability of the translations (including but not limited to local behavioural scores as they may be generalisable among different regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and populations and be unaffected by frequency of high-risk behav- adaptation or otherwise. iour. Unfortunately, our risk scores did not have promising Open access This is an open access article distributed in accordance with the characteristics and infections would be missed by using the tool. Creative Commons Attribution Non Commercial (CC BY- NC 4.0) license, which Various iterations to increase sensitivity or specificity did not permits others to distribute, remix, adapt, build upon this work non- commercially, lead to improved performance; this is probably due to the fact and license their derivative works on different terms, provided the original work is that many infectious syphilis cases are asymptomatic. properly cited, appropriate credit is given, any changes made indicated, and the use This study has several strengths. The data of the Centre for is non- commercial. See: http://creativecommons.org/licenses/by-nc/4.0/. Sexual Health of Amsterdam provided us with a large sample ORCID iDs size. Secondly, self-reported symptoms were systematically Silvia Achia Nieuwenburg http://orcid.org/0000-0003-1466-5292 noted in all the consultations. Furthermore, all clients were Elske Hoornenborg http://orcid.org/0000-0002-0512-2109 tested systematically based on their symptoms and risk behav- Henry John Christiaan de Vries http://orcid.org/0000-0001-9784-547X Maarten Schim van der Loeff http://orcid.org/0000-0002-4903-7002 iour. All assessed symptoms in this study were highly predictive for infectious syphilis. The study also has several limitations. We chose not to include risk behaviour in the risk score. Including REFERENCES risk behaviour might improve the algorithm, yet this would not 1 World Health Organization. Report on global sexually transmitted infection benefit our objective of self-reported symptoms- based algo- surveillance. Geneva: WHO, 2018. rithms leading to seeking care. Furthermore, the cut-offs of our 2 Hook EW. Syphilis. Lancet 2017;389:1550–7. risk scores were based on the Youden Index, but this is a rather 3 Wu MY, Gong HZ, Hu KR, et al. Effect of syphilis infection on HIV acquisition: a systematic review and meta- analysis. Sex Transm Infect 2021;97:525–33. arbitrary cut-off; the optimal cut- off may depend on the preva - 4 Clement ME, Okeke NL, Hicks CB. Treatment of syphilis: a systematic review. JAMA lence of infectious syphilis in the population, financial resources 2014;312:1905–17. and acceptability of low sensitivity. Lastly, we used data of MSM 5 Çakmak SK, Tamer E, Karadağ AS, et al. Syphilis: a great imitator. Clin Dermatol visiting the Centre for Sexual Health; possible results may not 2019;37:182–91. Nieuwenburg SA, et al. Sex Transm Infect 2022;0:1–6. doi:10.1136/sextrans-2022-055550 5 Original research 6 Ghanem KG, Ram S, Rice PA. The modern epidemic of syphilis. N Engl J Med and the potential role of syphilis immunity. Open Forum Infect Dis 2020;382:845–54. 2020;7:ofaa019. 7 Syfilis. Available: www.syfilistest.nl 14 Towns JM, Huffam S, Chow EPF, et al. Clinical factors associated with syphilis 8 Janier M, Unemo M, Dupin N, et al. 2020 European guideline on the management of concordance in men in sexual partnerships: a cross- sectional couples study. Sex syphilis. J Eur Acad Dermatol Venereol 2021;35:574–88. Transm Infect 2018;94:571–3. 9 Heymans R, van der Helm JJ, de Vries HJC, et al. Clinical value of Treponema pallidum 15 Dijkstra M, de Bree GJ, Stolte IG, et al. Development and validation of a risk score to real- time PCR for diagnosis of syphilis. J Clin Microbiol 2010;48:497–502. assist screening for acute HIV- 1 infection among men who have sex with men. BMC 10 Fluss R, Faraggi D, Reiser B. Estimation of the Youden index and its associated cutoff Infect Dis 2017;17:425. point. Biom J 2005;47:458–72. 16 Lin TC, Gianella S, Tenenbaum T, et al. A simple symptom score for acute human 11 Zhao P, Yang Z, Li B, et al. Simple-to- use nomogram for predicting the risk of syphilis immunodeficiency virus infection in a San Diego community- based screening program. among MSM in Guangdong Province: results from a serial cross- sectional study. BMC Clin Infect Dis 2018;67:105–11. Infect Dis 2021;21:1199. 17 Sanders EJ, Wahome E, Powers KA, et al. Targeted screening of at- risk adults for acute 12 Arando M, Fernandez-Nav al C, Mota- Foix M, et al. Early syphilis: risk factors and HIV- 1 infection in sub- Saharan Africa. AIDS 2015;29 Suppl 3:S221–30. clinical manifestations focusing on HIV- positive patients. BMC Infect Dis 18 Tordoff DM, Barbee LA, Khosropour CM, et al. Derivation and validation of an 2019;19:727. HIV risk prediction score among gay, bisexual, and other men who have sex with 13 Roth JA, Franzeck FC, Balakrishna S, et al. Repeated syphilis episodes in HIV-infected men to inform PreP initiation in an STD clinic setting. J Acquir Immune Defic Syndr men who have sex with men: a multicenter prospective cohort study on risk factors 2020;85:263–71. 6 Nieuwenburg SA, et al. Sex Transm Infect 2022;0:1–6. doi:10.1136/sextrans-2022-055550 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Sexually Transmitted Infections British Medical Journal

Developing a symptoms-based risk score for infectious syphilis among men who have sex with men

Loading next page...
 
/lp/british-medical-journal/developing-a-symptoms-based-risk-score-for-infectious-syphilis-among-YqdFCvHPvU

References (19)

Publisher
British Medical Journal
Copyright
© Author(s) (or their employer(s)) 2023. Re-use permitted under CC BY. Published by BMJ.
ISSN
1368-4973
eISSN
1472-3263
DOI
10.1136/sextrans-2022-055550
Publisher site
See Article on Publisher Site

Abstract

Original research Developing a symptoms- based risk score for infectious syphilis among men who have sex with men 1 1 1 Silvia Achia Nieuwenburg , Elske Hoornenborg , Udi Davidovich, 1,2,3 1,2,4 Henry John Christiaan de Vries , Maarten Schim van der Loeff ► Additional supplemental ABSTRACT WHAT IS ALREADY KNOWN ON THIS TOPIC material is published online Background Syphilis incidence is rising among men only. To view, please visit the ⇒ A symptoms- based risk score for infectious who have sex with men (MSM). An online tool based on journal online (http:// dx. doi. syphilis motivating men who have sex with men a risk score identifying men with higher risk of infectious org/ 10. 1136/ sextrans- 2022- (MSM) to seek care has not previously been 055550). syphilis could motivate MSM to seek care. We aimed developed. therefore to develop a symptoms- based risk score for Department of Infectious infectious syphilis. Diseases, Public Health Service WHAT THIS STUDY ADDS Methods We included data from all consultations by Amsterdam, Amsterdam, The ⇒ We developed a risk score based on symptoms Netherlands MSM attending the Amsterdam Centre for Sexual Health and notification for syphilis. Individual Amsterdam Institute for in 2018–2019. Infectious syphilis (ie, primary, secondary Infection and Immunity (AII), symptoms included in the risk score were or early latent syphilis) was diagnosed according to Amsterdam UMC location strongly associated with infectious syphilis, the centre’s routine protocol. Associations between University of Amsterdam, but the risk score had poor sensitivity and symptoms and infectious syphilis were expressed as odds Amsterdam, The Netherlands specificity. Department of Dermatology, ratios (OR), with 95% confidence intervals (CI). Based Amsterdam UMC location on multivariable logistic regression models, we created HOW THIS STUDY MIGHT AFFECT RESEARCH, University of Amsterdam, risk scores, combining various symptoms. We assessed Amsterdam, The Netherlands PRACTICE OR POLICY the area under the curve (AUC) and cut- off based on Department of Internal ⇒ Symptoms- based risk scores for infectious Medicine, Division of Infectious the Youden Index. We estimated which percentage of syphilis cannot be recommended to select MSM Diseases, Amsterdam UMC MSM should be tested based on a positive risk score and location University of for syphilis screening; all MSM at risk of syphilis which percentage of infectious syphilis cases would then Amsterdam, Amsterdam, the should be regularly screened. be missed. Netherlands Results We included 21,646 consultations with 11,594 unique persons. The median age was 34 years Correspondence to Silvia Achia Nieuwenburg, (IQR 27–45), and 14% were HIV positive (93% on in the early latent stage. Long-acting penicillin G, GGD Amsterdam, Amsterdam antiretroviral treatment). We diagnosed 538 cases of an effective and inexpensive antibiotic, remains the 1018 WT, The Netherlands; infectious syphilis. Associations with syphilis symptoms/ snieuwenburg@ ggd. recommended treatment for syphilis. signs were strong and highly significant, for example, OR amsterdam. nl The signs and symptoms of syphilis vary for a painless penile ulcer was 35.0 (CI 24.9 to 49.2) and depending on the stage, and for this reason syphilis Received 15 June 2022 OR for non- itching rash 57.8 (CI 36.8 to 90.9). Yet, none is also known as ‘the great imitator’. This makes it Accepted 21 October 2022 of the individual symptoms or signs had an AUC >0.55. challenging for patients to recognise the disease and The AUC of risk scores combining various symptoms seek a health provider when they are symptomatic. varied from 0.68 to 0.69. For all risk scores using cut- offs The primary stage is characterised by a usually pain- based on Youden Index, syphilis screening would be less ulcer at the site of infection, usually appearing recommended in 6% of MSM, and 59% of infectious within 3 months after acquisition and disappearing syphilis cases would be missed. within 6 weeks. If left untreated, the disease may Conclusion Symptoms- based risk scores for infectious progress to the secondary stage characterised by syphilis perform poorly and cannot be recommended to a non- itching rash. Early latent stage syphilis is select MSM for syphilis screening. All MSM with relevant asymptomatic and can be diagnosed with serolog- sexual exposure should be regularly tested for syphilis. ical tests in patients without a history of primary or secondary symptoms. © Author(s) (or their employer(s)) 2022. Re- use The increasing incidence of syphilis calls for permitted under CC BY- NC. No INTRODUCTION preventive efforts to reduce exposure to the infec- commercial re- use. See rights Syphilis, caused by the bacterium Treponema pall- tion. To reduce risk of ongoing transmission (espe- and permissions. Published idum, remains a public health concern. In many cially in the primary and secondary stage) and of by BMJ. countries, the incidence of syphilis is rising, espe- sequelae, early diagnosis and treatment are key, To cite: Nieuwenburg SA, cially among populations of men who have sex with and it would be helpful if patients would be able to Hoornenborg E, men (MSM). If left untreated, it can lead to serious recognise the disease and be motivated to seek care. Davidovich U, et al. cardiovascular and neurological complications. Therefore, the Centre for Sexual Health of Sex Transm Infect Epub ahead of print: [please include Day Syphilis can also facilitate the transmission of HIV. the Public Health Service of Amsterdam and the Month Year]. doi:10.1136/ It is highly contagious through sexual contact national non- governmental organisation STI- sextrans-2022-055550 during the primary and secondary stage, as well as AIDS Netherlands launched a campaign to create Nieuwenburg SA, et al. Sex Transm Infect 2022;0:1–6. doi:10.1136/sextrans-2022-055550 1 Original research Table 1 Characteristics of all MSM at first consultation in study period; Amsterdam Centre for Sexual Health, July 2018–July 2019 Total Infectious syphilis No infectious syphilis Variables (n=11,594) (n=274) (n=11,320) P value Age in years (median, IQR) 34 (27–45) 36 (29–47) 34 (27–45) 0.009 Country of birth, n (%) Netherlands 7,332 (63.2) 155 (56.6) 7,177 (63.4) 0.020 Othe4,r 4,262 (36.8) 119 (43.4) 4,143 (36.6) Sexual behaviour, n (%) MSM 10,237 (88.3) 260 (94.9) 9,977 (88.1) 0.001 MSMW 1,357 (11.7) 14 (5.1) 1,343 (11.9) HIV status, n (%) Negative 10,005 (86.3) 152 (55.5) 9,853(87.0) <0.001 Positive 1,586 (13.7) 122 (44.5) 1,464 (12.9) Unknown 3 (0.0) 3 (0.0) 0 (0.0) On ART, n (%)* 1,396 (93.0) 103 (89.6) 1,293 (93.3) 0.132 HIV VL as reported by patient, n (%) Undetectable 1,379 (86.9) 105 (86.1) 1,274 (87.0) <0.001 Detectable 33 (2.1) 3 (2.5) 30 (2.0) Notified for syphilis 421 (3.6) 56 (20.4) 365 (3.2) <0.001 STD diagnosis at current visit, n (%) Chlamydia 1,146 (9.9) 52 (19.0) 1,094 (9.7) <0.001 Gonorrhoea 1,290 (11.1) 62 (22.6) 1,228 (10.9) <0.001 Lymphogranuloma venereum 73 (0.6) 8 (2.9) 65 (0.6) <0.001 Hepatitis B 9 (0.1) 1 (0.4) 8 (0.1) 0.084 Hepatitis C 3 (0) 0 (0) 3 (0) 0.788 *For those who are HIV positive, data on ART use of 85 participants were missing. ART, antiretroviral therapy; IQR, interquartile range; MSM, men who have sex with men; MSMW, men who have sex with men and women; STD, sexually transmitted disease; VL, viral load. awareness of syphilis in Amsterdam, the Netherlands. The syphilis. A syphilis diagnosis was made according to the centre’s awareness campaign, which ran from 2018 to 2019, included routine protocol using a combination of clinical signs and symp- an online tool to enable individuals to self- identify symptoms toms and laboratory testing based on the European guidelines that could be related to syphilis, and if indicated motivate them for syphilis management. Serological screening for syphilis to seek care (testing and immediate treatment in case a diagnosis consisted of a treponemal test chemiluminescence immunoassay was made). This online tool included an expert- based algorithm (CLIA, DiaSorin, Saluggia, Italy) confirmed by an immunoblot of various symptoms; we also included a partner notification (INNO- LIA Syphilis, Innogenetics, Ghent, Belgium) and a non- for syphilis in the risk score. The aim of the current study was treponemal test, the rapid plasma reagin (RPR, RPR NOSTICON to assess whether an evidence- based symptoms- based risk score II, bioMérieux, Marcy l’Etoile, France). In persons with suspected could be developed, which could help individuals to self- identify primary syphilis, we additionally performed dark- field micros- possible infectious syphilis. copy of ulcer exudate and a PCR of ulcer scraping. Secondary syphilis was diagnosed if a client had a positive serology with characteristic skin or mucocutaneous lesions. Early latent syph- METHODS ilis was defined as confirmed positive serology without clinical There are few data from the peer-reviewed literature on the asso- manifestations, according to the European guidelines. ciations of symptoms with a current syphilis infection. We there- We described person characteristics by medians and inter- fore developed a syphilis assessment algorithm in consultation quartile ranges (IQRs), and numbers and percentages. Univari- with experts (dermatologists and internist- infectiologist). Based able logistic regression was done using Generalized Estimating on the expert opinions, the following signs and symptoms were Equations (GEE), because of correlated data (multiple visits by included in the risk algorithm: a painful or painless ulcer on the the same person). Logistic regression was done to obtain ORs penis, in the mouth, at the anus or on the skin, with or without with 95% CIs for infectious syphilis diagnosis for each symptom palpable regional lymph nodes, and an itching or non- itching and risk factor. Various multivariable models were constructed rash with or without influenza-like symptoms. In addition, we to enable creation of a series of prediction models and predic- also added partner notification for syphilis. A complete overview tion scores. Our initial risk score A1 was based on the symptoms of the included items is provided in online supplemental table 1. of the expert- based algorithm: ulcer in the mouth, ulcer on the To assess the validity of the expert-based algorithm, we used skin, painless ulcer at the anus with palpable regional lymph data from all consultations by MSM attending the Centre for nodes, painless ulcer on the penis with palpable regional lymph Sexual Health of the Amsterdam Public Health Service, the nodes, painless ulcer at the anus without palpable regional lymph Netherlands, from July 2018 to July 2019, regardless of reason nodes, painless ulcer on the penis without palpable regional of visit. Consultations were at the client’s own initiative and free lymph nodes, painful ulcer at the anus with palpable regional of charge. lymph nodes, painful ulcer on the penis with palpable regional For this study, the outcome of interest was infectious syph- lymph nodes, painful ulcer at the anus without palpable regional ilis, which comprises primary, secondary and early latent 2 Nieuwenburg SA, et al. Sex Transm Infect 2022;0:1–6. doi:10.1136/sextrans-2022-055550 Original research Table 2 Multivariable analysis of possible syphilis symptoms and their association with a diagnosis of infectious syphilis among MSM; Amsterdam Centre for Sexual Health, 2018–2019 Model A* Model B† Model C‡ Coding Symptoms/risk factor aOR (95% CI) βˆ aOR (95% CI) βˆ aOR (95% CI) βˆ 1.m Ulcer in the mouth No Ref Yes 3.2 (0.9 to 12.0) 1.2 1.s Ulcer on the skin§ No Ref Yes 0.7 (0.1 to 4.4) −0.3 1.1.p Painless ulcer on the penis No Ref Yes 36.1 (25.2 to 51.6) 3.6 1.1.1.a Painless ulcer at the anus with lymph nodes¶ No Ref Ref Ref Yes 63.8 (4.1 to 991.8) 4.2 63.6 (4.1 to 989.3) 4.2 65.4 (4.4 to 981.1) 4.2 1.1.1.p Painless ulcer on the penis with lymph nodes No Ref Ref Yes 85.8 (36.6 to 201.0) 4.5 85.7 (36.6 to 200.6) 4.5 1.1.2.a Painless ulcer at the anus without lymph nodes No Ref Yes 1.4 (0.3 to 6.5) 0.4 1.1.2.p Painless ulcer on the penis without lymph nodes No Ref Ref Yes 29.1 (19.5 to 43.4) 3.7 30.0 (20.2 to 44.7) 3.4 1.2.p Painful ulcer on the penis No Ref Yes 21.7 (14.2 to 33.2) 3.1 1.2.1.a Painful ulcer at the anus with lymph nodes No Ref Yes 1.5 (0.2 to 11.7) 0.4 1.2.1.p Painful ulcer on the penis with lymph nodes No Ref Ref Yes 11.5 (4.1 to 32.0) 2.4 13.0 (4.9 to 34.4) 2.6 1.2.2.a Painful ulcer at the anus without lymph nodes No Ref Ref Ref Yes 9.6 (4.9 to 18.6) 2.3 9.6 (5.0 to 18.7) 2.3 9.7 (5.0 to 18.8) 2.3 1.2.2.p Painful ulcer on the penis without lymph nodes No Ref Ref Yes 25.0 (15.6 to 39.9) 3.2 24.7 (15.5 to 39.6) 3.2 2.paso Rash on the palms of the hands or soles of the feet No Ref Yes 2.6 (0.8 to 8.1) 2.1 Itching rash¶ No Ref Ref Ref Yes 7.4 (3.4 to 16.3) 2.0 8.1 (3.8 to 17.2) 2.1 8.0 (3.7 to 17.0) 2.1 2.2Non- itching rash No Ref Ref Ref Yes 54.3 (32.7 to 90.2) 4.0 64.8 (40.4 to 103.7) 4.2 64.8 (40.5 to 103.7) 4.2 3. Notified for syphilis No Ref Ref Ref Yes 4.4 (3.3 to 5.7) 1.5 4.4 (3.3 to 5.7) 1.5 4.3 (3.3 to 5.7) 1.5 β∧ is the regression coefficient of the logistic regression models (i.e. the log of the Odds Ratio). *Model A is based on the symptoms of the expert- based algorithm: ulcer in the mouth, ulcer on the skin, painless ulcer at the anus with palpable regional lymph nodes, painless ulcer on the penis with palpable regional lymph nodes, painless ulcer at the anus without palpable regional lymph nodes, painless ulcer on the penis without palpable regional lymph nodes, painful ulcer at the anus with palpable regional lymph nodes, painful ulcer on the penis with palpable regional lymph nodes, painful ulcer at the anus without palpable regional lymph nodes, painful ulcer on the penis without palpable regional lymph nodes, rash on the palms of the hands or soles of the feet, itching rash, non-itching rash and a partner notification for syphilis. †Model B is based on the symptoms and risk factor (a partner notification for syphilis) that were significantly associated with infectious syphilis: painless ulcer at the anus with palpable regional lymph nodes, painful ulcer at the anus without palpable regional lymph nodes, painless ulcer on the penis with palpable regional lymph nodes, painless ulcer on the penis without palpable regional lymph nodes, painful ulcer on the penis with palpable regional lymph nodes, painful ulcer on the penis without palpable regional lymph nodes, itching rash, non- itching rash and a partner notification for syphilis. ‡Model C is based on model B, but simplified by including painless and painful ulcers (both regardless of lymph nodes): painless ulcer at the anus with palpable regional lymph nodes, painful ulcer at the anus without palpable regional lymph nodes, painless ulcer on the penis, painful ulcer on the penis, itching rash, non-itching rash and a partner notification for syphilis. §Skin is defined as location other than anogenital (anus or penis) or mouth. ¶Rash associated with syphilis: a maculopapular exanthema or erythematous exanthema. aOR, adjusted OR; MSM, men who have sex with men. lymph nodes, painful ulcer on the penis without palpable score. Here we only included the symptoms and risk factor (a regional lymph nodes, rash on the palms of the hands or soles of partner notification for syphilis) that were significantly asso- the feet, itching rash, non- itching rash and a partner notification ciated with infectious syphilis in model A1, without weighing. for syphilis (online supplemental table 1). Risk score B2 included the same factors as risk score B1, but In risk score A1, each factor was awarded 1 point if present with weighing based on the regression coefficients. To optimise and 0 if not, that is, without weighing. Risk score A2 included the risk scores, we also calculated risk scores C1 and C2. Penile the same factors as risk score A1, but here each symptom/factor ulcers were associated with infectious syphilis, independently was given a weight equal to the rounded regression coefficient of lymph nodes being palpable or not. Therefore, risk score of the multivariable analysis. Risk score B1 was a simplified risk C1 included the same factors as risk score B1, but rather than Nieuwenburg SA, et al. Sex Transm Infect 2022;0:1–6. doi:10.1136/sextrans-2022-055550 3 Original research RESULTS In total, 21,646 consultations of 11,594 persons were included in this study (see table 1 which provides details of persons at their first visit in the study period). The median age was 34 years. A total of 1,586 (13.7%) were living with HIV; of the 1,501 with information on viral load, 1,379 (91.9%) reported an undetectable viral load. In total, 421 (3.6%) persons were notified for syphilis. At the 21,646 consultations, 538 cases of infectious syphilis were diagnosed (2.5%). A total of 1270 men (5.9%) reported at least one of the symptoms included in the expert-based risk score. In 260 consultations (1.2%), a penile ulcer was present, in 114 (0.5%) an anal ulcer and in 148 (0.7%) a skin rash. The characteristic rash on hand palms or feet soles was rare, present in 23 (0.1%) consultations. In univariable analysis, all symptoms and the risk factor (a partner notification for syphilis) used were associated with a Figure 1 ROC curve of score C2 among MSM with infectious syphilis; diagnosis of infectious syphilis (online supplemental table 2). Amsterdam Centre for Sexual Health. Risk score C2 is comprised of the The self- reported symptoms: non- itching rash with influenza- following symptoms (between brackets the weights of each symptom): like symptoms (OR 86.8, 95% CI 35.2 to 214.1), rash on the painless anal ulcer with palpable regional lymph nodes (4.2)+painful soles of the feet (OR 78.6, 95% CI 7.1 to 867.6), painless anal anal ulcer with no palpable regional lymph nodes (2.3)+painless penile ulcer with palpable regional lymph nodes (OR 78.6, 95% CI 7.1 ulcer (3.5)+painful penile ulcer (3.1)+itching rash (2.1)+non-itching to 867.6) and painless penile ulcer with palpable regional lymph rash (4.2)+a partner notification for syphilis (1.5). MSM, men who have nodes (OR 76.6, 95% CI 34.0 to 172.6) were most strongly sex with men; ROC, receiver operating characteristics. associated with an infectious syphilis diagnosis. The AUCs of the individual symptoms or signs were between 0.50 and 0.55. In a multivariable analysis including all symptoms (and a including painless and painful ulcers with and without regional partner notification for syphilis) included in the expert-based palpable lymph nodes, we included painless and painful ulcers, risk score (model A), the following symptoms were strongly asso- both regardless of lymph nodes. Risk score C2 included the same ciated with an infectious syphilis diagnosis: painless penile ulcer factors as risk score C1, but with weighing based on the regres- with palpable regional lymph nodes (adjusted OR (aOR) 85.8, sion coefficients. 95% CI 36.6 to 201.0), painless anal ulcer with palpable regional For each risk score, we assessed the area under the curve lymph nodes (aOR 63.8, 95% CI 4.1 to 991.8) and non-itching (AUC). To evaluate the clinical relevance of the risk scores A, rash (aOR 54.3, 95% CI 32.7 to 90.2) (table 2). In model B, B and C, the post- test probability of disease was evaluated. The which is similar to model A, but with non-significant symptoms positive predictive value of having infectious syphilis when omitted, the same symptoms were strongly associated with an having a score above the cut- off was estimated; the cut- off was infectious syphilis diagnosis: painless penile ulcer with palpable based on the Youden Index. The Youden Index measures the effectiveness of a diagnostic marker as follows: sensitivity+spec- regional lymph nodes (aOR 85.7, 95% CI 36.6 to 200.6), non- ificity–1. As there is no generally agreed optimum of sensitivity itching rash (aOR 64.8, 95% CI 40.4 to 103.7) and painless anal and specificity combination, we selected the highest value for ulcer with palpable regional lymph nodes (aOR 63.6, 95% CI the Youden Index as optimal cut-off . Furthermore, we estimated 4.1 to 989.3). Because a painless penile ulcer was strongly asso- which percentage of infectious syphilis cases would be missed, if ciated with infectious syphilis regardless of palpable regional patients would only be tested for syphilis if their score exceeded lymph nodes, and also a painful penile ulcer was strongly asso- the cut-off of the risk score. A p value of <0.05 was consid- ciated with infectious syphilis regardless of palpable regional ered statistically significant. We carried out analyses using Stata lymph nodes, we made model C in which ‘painless penile ulcer’ (V.15.1, StataCorp, College Station, Texas, USA). and ‘painful penile ulcer’ were included rather than the four Table 3 Performance of several risk scores for infectious syphilis among MSM; Amsterdam Centre for Sexual Health, July 2018–July 2019 Infectious syphilis among Infectious syphilis among those with a score of at those with a score below Sensitivity Specificity % to be Risk score*Cut- off† least the cut- offthe cut- off (95% CI) (95% CI) AUC tested Score A1 1.0 220/1,270 318/20,376 40.9 (36.7 to 45.2) 95.0 (94.7 to 95.3) 0.6811 5.9 Score A2 0.9 220/1,226 318/20,420 40.9 (36.7 to 45.2) 95.2 (94.9 to 95.5) 0.6846 5.7 Score B1 1.0 218/1,206 320/20,440 40.5 (36.3 to 44.8) 95.3 (95.0 to 95.6) 0.6811 5.6 Score B2 1.5 218/1,206 320/20,440 40.5 (36.3 to 44.8) 95.3 (95.0 to 95.6) 0.6846 5.6 Score C1 1.0 218/1,206 320/20,440 40.5 (36.3 to 44.8) 95.3 (95.0 to 95.6) 0.6811 5.6 Score C2 1.5 218/1,206 320/20,440 40.5 (36.3 to 44.8) 95.3 (95.0 to 95.6) 0.6846 5.6 *Risk scores are based on the multivariable models as explained in table 2. Risk scores A1, B1 and C1 are based on a summation of all risk factors, each with identical weight (one). Risk scores A2, B2 and C2 are based on a summation of the weights of all risk factors, the weight being identical to the regression coefficient of a factor in the multivariable model (see table 2). †Based on the Youden Index. AUC, area under the curve; MSM, men who have sex with men. 4 Nieuwenburg SA, et al. Sex Transm Infect 2022;0:1–6. doi:10.1136/sextrans-2022-055550 Original research separate items. In this optimised model C, a painless anal ulcer be generalisable to all MSM including those who do not yet seek with inguinal lymph nodes (aOR 65.4, 95% CI 4.4 to 981.1), sexual healthcare. non- itching rash (aOR 64.8, 95% CI 40.5 to 103.7) and painless For the prevention of syphilis, multiple approaches are penile ulcer (aOR 36.1, 95% CI 25.2 to 51.6) were most strongly necessary. An awareness campaign with a symptoms- based risk associated with an infectious syphilis diagnosis. score will likely not prevent the ongoing transmission of infec- Risk score A1, based on the expert-based algorithm, had a tious syphilis. Early identification of the infection and timely sensitivity of 40.9% and a specificity of 95.0%. Based on the treatment are helpful to reduce the duration of infection and cut- off of 1.0 (based on Youden Index), 5.9% of men would onwards sequela. An online symptoms tool might in some cases have to be tested and 59% of syphilis cases would remain unde- be harmful as it may warrant incorrect assessment of no risk. tected. The risk score using weighing (A2) performed similarly. Thus, all MSM who engage in sexual behaviour that places them The other four risk scores (B1, B2, C1, C2) performed almost at higher risk of acquiring STIs should be regularly tested for identical to the risk scores A1 and A2. The AUC was 0.68 for syphilis. Yet, an online tool might help to motivate MSM to seek risk score A1 based on the expert- opinion algorithm (figure 1). help timely in case of suspected symptoms. The AUCs of the other risk scores, including our optimised risk In conclusion, our proposed symptoms- based risk scores score C2, were all also 0.68. Table 3 shows the sensitivity and performed poorly to diagnose infectious syphilis. Thus, we specificity of the various risk scores for infectious syphilis. cannot recommend a symptoms- based risk score to select MSM for syphilis screening. All MSM with relevant sexual exposure should be regularly tested for syphilis. DISCUSSION We aimed to assess whether an evidence-based symptoms- based Handling editor Nigel Field risk score could be developed to identify individuals with infec- Twitter Henry John Christiaan de Vries @henryjdevries tious syphilis. The optimised risk score C2, including six symp- Contributors MSvdL, HJCdV and SN conceptualised and designed the study. SN toms and a partner notification for syphilis, had a sensitivity of performed the data analysis and interpretation of the data. MSvdL reviewed all 40.5% (95% CI 36.3% to 44.8%), a specificity of 95.3% (95% analyses. SN drafted the manuscript. All authors critically revised the manuscript. MSvdL had full responsibility for the work, the conduct of the study, had access to CI 95.0% to 95.6%) and an AUC of 0.68. Based on this score, the data, and controlled the decision to publish. 5.6% of the study population would be eligible for syphilis Funding The authors have not declared a specific grant for this research from any testing and 41% of the cases would be identified. funding agency in the public, commercial or not- for-profit sectors . We had aimed that a symptoms- based risk score would enable Competing interests None declared. MSM to self-identify possible infectious syphilis to increase early testing and diagnosis, and avoid further transmission and compli- Patient consent for publication Not required. cations. To our knowledge, there are no studies on the develop- Ethics approval Ethical approval was not required for this study, as it made use of ment of a symptoms-based risk score for infectious syphilis. One routinely collected data and all data were de- identified prior to analysis. study developed a simple-to- use nomogram to predict the risk Provenance and peer review Not commissioned; externally peer reviewed. of syphilis, and several other studies described the risk factors Data availability statement All data relevant to the study are included in the for syphilis based on sociodemographic, clinical or behavioural article or uploaded as supplemental information. 11–14 data. The main objective of the current study was to develop Supplemental material This content has been supplied by the author(s). a symptoms- based risk score so that individuals would seek help It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not when symptomatic, analogous to similar algorithms for acute have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all HIV infection. Several symptom and behavioural risk scores for 15–18 liability and responsibility arising from any reliance placed on the content. acute HIV have been developed and validated successfully. Where the content includes any translated material, BMJ does not warrant the The symptoms- based risk scores may have the advantage above accuracy and reliability of the translations (including but not limited to local behavioural scores as they may be generalisable among different regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and populations and be unaffected by frequency of high-risk behav- adaptation or otherwise. iour. Unfortunately, our risk scores did not have promising Open access This is an open access article distributed in accordance with the characteristics and infections would be missed by using the tool. Creative Commons Attribution Non Commercial (CC BY- NC 4.0) license, which Various iterations to increase sensitivity or specificity did not permits others to distribute, remix, adapt, build upon this work non- commercially, lead to improved performance; this is probably due to the fact and license their derivative works on different terms, provided the original work is that many infectious syphilis cases are asymptomatic. properly cited, appropriate credit is given, any changes made indicated, and the use This study has several strengths. The data of the Centre for is non- commercial. See: http://creativecommons.org/licenses/by-nc/4.0/. Sexual Health of Amsterdam provided us with a large sample ORCID iDs size. Secondly, self-reported symptoms were systematically Silvia Achia Nieuwenburg http://orcid.org/0000-0003-1466-5292 noted in all the consultations. Furthermore, all clients were Elske Hoornenborg http://orcid.org/0000-0002-0512-2109 tested systematically based on their symptoms and risk behav- Henry John Christiaan de Vries http://orcid.org/0000-0001-9784-547X Maarten Schim van der Loeff http://orcid.org/0000-0002-4903-7002 iour. All assessed symptoms in this study were highly predictive for infectious syphilis. The study also has several limitations. We chose not to include risk behaviour in the risk score. Including REFERENCES risk behaviour might improve the algorithm, yet this would not 1 World Health Organization. Report on global sexually transmitted infection benefit our objective of self-reported symptoms- based algo- surveillance. Geneva: WHO, 2018. rithms leading to seeking care. Furthermore, the cut-offs of our 2 Hook EW. Syphilis. Lancet 2017;389:1550–7. risk scores were based on the Youden Index, but this is a rather 3 Wu MY, Gong HZ, Hu KR, et al. Effect of syphilis infection on HIV acquisition: a systematic review and meta- analysis. Sex Transm Infect 2021;97:525–33. arbitrary cut-off; the optimal cut- off may depend on the preva - 4 Clement ME, Okeke NL, Hicks CB. Treatment of syphilis: a systematic review. JAMA lence of infectious syphilis in the population, financial resources 2014;312:1905–17. and acceptability of low sensitivity. Lastly, we used data of MSM 5 Çakmak SK, Tamer E, Karadağ AS, et al. Syphilis: a great imitator. Clin Dermatol visiting the Centre for Sexual Health; possible results may not 2019;37:182–91. Nieuwenburg SA, et al. Sex Transm Infect 2022;0:1–6. doi:10.1136/sextrans-2022-055550 5 Original research 6 Ghanem KG, Ram S, Rice PA. The modern epidemic of syphilis. N Engl J Med and the potential role of syphilis immunity. Open Forum Infect Dis 2020;382:845–54. 2020;7:ofaa019. 7 Syfilis. Available: www.syfilistest.nl 14 Towns JM, Huffam S, Chow EPF, et al. Clinical factors associated with syphilis 8 Janier M, Unemo M, Dupin N, et al. 2020 European guideline on the management of concordance in men in sexual partnerships: a cross- sectional couples study. Sex syphilis. J Eur Acad Dermatol Venereol 2021;35:574–88. Transm Infect 2018;94:571–3. 9 Heymans R, van der Helm JJ, de Vries HJC, et al. Clinical value of Treponema pallidum 15 Dijkstra M, de Bree GJ, Stolte IG, et al. Development and validation of a risk score to real- time PCR for diagnosis of syphilis. J Clin Microbiol 2010;48:497–502. assist screening for acute HIV- 1 infection among men who have sex with men. BMC 10 Fluss R, Faraggi D, Reiser B. Estimation of the Youden index and its associated cutoff Infect Dis 2017;17:425. point. Biom J 2005;47:458–72. 16 Lin TC, Gianella S, Tenenbaum T, et al. A simple symptom score for acute human 11 Zhao P, Yang Z, Li B, et al. Simple-to- use nomogram for predicting the risk of syphilis immunodeficiency virus infection in a San Diego community- based screening program. among MSM in Guangdong Province: results from a serial cross- sectional study. BMC Clin Infect Dis 2018;67:105–11. Infect Dis 2021;21:1199. 17 Sanders EJ, Wahome E, Powers KA, et al. Targeted screening of at- risk adults for acute 12 Arando M, Fernandez-Nav al C, Mota- Foix M, et al. Early syphilis: risk factors and HIV- 1 infection in sub- Saharan Africa. AIDS 2015;29 Suppl 3:S221–30. clinical manifestations focusing on HIV- positive patients. BMC Infect Dis 18 Tordoff DM, Barbee LA, Khosropour CM, et al. Derivation and validation of an 2019;19:727. HIV risk prediction score among gay, bisexual, and other men who have sex with 13 Roth JA, Franzeck FC, Balakrishna S, et al. Repeated syphilis episodes in HIV-infected men to inform PreP initiation in an STD clinic setting. J Acquir Immune Defic Syndr men who have sex with men: a multicenter prospective cohort study on risk factors 2020;85:263–71. 6 Nieuwenburg SA, et al. Sex Transm Infect 2022;0:1–6. doi:10.1136/sextrans-2022-055550

Journal

Sexually Transmitted InfectionsBritish Medical Journal

Published: Aug 18, 2023

Keywords: syphilis; diagnosis; homosexuality, male

There are no references for this article.