Access the full text.
Sign up today, get DeepDyve free for 14 days.
APRIL 2023 HUA N G E T A L . 455 a a b b c d BOYIN HUANG , XUNGANG YIN, JAMES A. CARTON, LIGANG CHEN, GARRETT GRAHAM, CHUNYING LIU, e a THOMAS SMITH, AND HUAI-MIN ZHANG NOAA/National Centers for Environmental Information, Asheville, North Carolina Department of Atmospheric and Oceanic Sciences, University of Maryland, College Park, College Park, Maryland North Carolina Institute for Climate Studies, North Carolina State University, Asheville, North Carolina Riverside Technology, Inc., Asheville, North Carolina NOAA/Center for Satellite Applications and Research, College Park, Maryland (Manuscript received 25 July 2022, in ﬁnal form 18 January 2023, accepted 26 January 2023) ABSTRACT: Our study shows that the intercomparison among sea surface temperature (SST) products is inﬂuenced by the choice of SST reference, and the interpolation of SST products. The inﬂuence of reference SST depends on whether the reference SSTs are averaged to a grid or in pointwise in situ locations, including buoy or Argo observations, and ﬁltered by ﬁrst-guess or climatology quality control (QC) algorithms. The inﬂuence of the interpolation depends on whether SST products are in their original grids or preprocessed into common coarse grids. The impacts of these factors are demon- strated in our assessments of eight widely used SST products (DOISST, MUR25, MGDSST, GAMSSA, OSTIA, GPB, CCI, CMC) relative to buoy observations: (i) when the reference SSTs are averaged onto 0.258 3 0.258 grid boxes, the mag- nitude of biases is lower in DOISST and MGDSST (,0.038C), and magnitude of root-mean-square differences (RMSDs) is lower in DOISST (0.388C) and OSTIA (0.438C); (ii) when the same reference SSTs are evaluated at pointwise in situ locations, the standard deviations (SDs) are smaller in DOISST (0.388C) and OSTIA (0.398C) on 0.258 3 0.258 grids; but the SDs become smaller in OSTIA (0.348C) and CMC (0.378C) on products’ original grids, showing the advantage of those high-resolution analyses for resolving ﬁner-scale SSTs; (iii) when a loose QC algorithm is applied to the reference buoy observations, SDs increase; and vice versa; however, the relative performance of products remains the same; and (iv) when the drifting-buoy or Argo observations are used as the reference, the magnitude of RMSDs and SDs become smaller, potentially due to changes in observing intervals. These results suggest that high-resolution SST analyses may take advan- tage in intercomparisons. SIGNIFICANCE STATEMENT: Intercomparisons of gridded SST products be affected by how the products are compared with in situ observations: whether the products are in coarse (0.258)ororiginal (0.058–0.108) grids, whether the in situ SSTs are in their reported locations or gridded and how they are quality controlled, and whether the biases of satellite SSTs are corrected by localized matchups or large-scale patterns. By taking all these factors into account, our analyses indicate that the NOAA DOISST is among the best SST products for the long period (1981–present) and relatively coarse (0.258) resolution that it was designed for. KEYWORDS: Sea surface temperature; In situ oceanic observations; Satellite observations; Bias 1. Introduction (Yates et al. 2016; Singels and Bezuidenhout 1999), coastal watch (Miller and DeCampo 1994; Lima and Wethey 2012; Cole Sea surface temperature (SST) as an important climate indica- 2000), climatic impacts at decadal and multidecadal time scales tor has numerous applications at different spatial and temporal (Mohino et al. 2011; Sun et al. 2016; Vibhute et al. 2020), and scales. For example, SSTs are used in studying short-term ex- long-term global warming (Karl et al. 2015; Zhang 2016). Many treme weather events (Feudale and Shukla 2011; Hartmann of these applications require an accurate gridded product for 2015), extreme marine heatwave events (Hobday et al. 2016; weather and ocean forecasting (O’Carroll et al. 2019,and refer- Huang et al. 2021c; D’Agata 2022), impacts of El Niño and ences therein), climate projections (He and Soden 2016), and Southern Oscillation (ENSO) on ﬁsheries and agriculture coastal watch (Shimada et al. 2015). To meet these applications’ requirements, SST products with various spatial (0.018–58)and temporal (6 h, daily, and monthly) resolutions have been devel- Denotes content that is immediately available upon publica- oped based on in situ observations from ships, buoys and Argo tion as open access. ﬂoats, and satellites (Huang et al. 2017, and references therein). The quality of gridded SST products is usually assessed by Supplemental information related to this paper is available at comparing to a reference SST. An ensemble of available SST the Journals Online website: https://doi.org/10.1175/JTECH-D-22- products may be used as a reference (Dash et al. 2012; Chin 0081.s1. et al. 2017; Yang et al. 2021; Huang et al. 2021b), e.g., the Group for High Resolution SST (GHRSST) Multiproduct Corresponding author: Boyin Huang, firstname.lastname@example.org Ensemble (GMPE; Martin et al. 2012; Fiedler et al. 2019). DOI: 10.1175/JTECH-D-22-0081.1 Ó 2023 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses). 456 J OUR N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L OGY VOLUME 40 The in situ SSTs from drifting buoys, moored buoys, and (VIIRS). These observed SSTs are ﬁltered by the QC (see more Argo ﬂoats were frequently used as an ideal reference due to details in section 2c)of the ﬁrst guess (FG), which is the their accuracy and high spatial and temporal coverages DOISST analysis in the previous day. The biases of satellite (Huang et al. 2021b; also, in this study). The SSTs derived SSTs were quantiﬁed by the difference between large-scale pat- from conductivity–temperature–depth (CTD) can be used as terns of satellite and in situ SSTs within 3000 km in latitude, a reference, but their accuracy may be impacted by depth 5000 km in longitude, and 15-day data window, which were de- errors and near-surface contaminations (Huang et al. 2018; termined by the empirical orthogonal teleconnection functions Moteki 2022). The SST measurements from modern sail- (EOTs; Reynolds et al. 2007). drones and thermosalinographs are accurate but spatial and temporal coverages are limited, which can be used as referen- 2) MUR25 ces in regional assessment (Vazquez-Cuervo et al. 2022). The NASA Multiscale Ultrahigh Resolution (MUR25) v4.1 The reference SST can be processed either to the GMPE analysis is a daily 0.258 3 0.258SST product starting from 2002 grids (0.258 3 0.258; a common grid established for the sake of (Chin et al. 2017). MUR v4.1 includes in situ SSTs from the intercomparison) (Huang et al. 2021a,b) or in its in situ loca- tions (Dash et al. 2012; Fiedler et al. 2019), and may be ﬁl- NOAA iQuam project (Xu and Ignatov 2010), which includes SSTs from ships, drifting and moored buoys, and Argo ﬂoats. tered by different quality-control (QC) algorithms (Reynolds The in situ SSTs were blended with nighttime SSTs derived et al. 2007; Dash et al. 2010). Likewise, the gridded SST prod- from AVHRR, Advanced Microwave Scanning Radiometer- ucts can be processed either to the GMPE grids (Martin et al. 2012; Fiedler et al. 2019; Yang et al. 2021; Huang et al. 2021b, EOS (AMSR-EOS), AMSR2, the Moderate Resolution Imaging 2017) or in their original grids (Dash et al. 2012; Fiedler et al. Spectroradiometers (MODIS), and the U.S. Navy microwave 2019). WindSat radiometer. Biases in satellite SSTs are adjusted accord- A recent assessment (Huang et al. 2021b) indicated that the ing toinsituSSTs. NOAA Daily Optimum Interpolation (OI) SST (DOISST) v2.1 has a good performance, while the NOAA SST Quality 3) MGDSST Monitor (SQUAM; Fig. S1) showed that Met Ofﬁce Opera- The Japan Meteorological Agency (JMA) Merged satellite tional SST and Sea Ice Analysis (OSTIA; Good et al. 2020) and in situ data Global Daily Sea Surface Temperature has a better performance. There are questions about whether (MGDSST) is a daily 0.258 3 0.258 product starting from 1982 the intercomparisons are sensitive to these details of the com- to 2020 (Kurihara et al. 2006). The MGDSST includes in situ parisons and how we understand the differences, which are SST from buoys and ships, satellite SSTs retrieved from infrared subjects of this paper. sensors (NOAA/AVHRR, MetOp/AVHRR), microwave sen- In this study, we address the reasons for the differences sors (Coriolis/WindSat, GCOM-W1/AMSR-2, Aqua/AMSR-E), among the SST intercomparisons between nine widely used and ACSPO version 2.60 after December 2018 (Sakurai et al. daily gridded SST products (section 2a). The intercomparison 2019). methods are described in section 2b.In section 3a,we show why an ensemble SST reference is not preferable. In section 3b, we demonstrate how intercomparisons are inﬂuenced by 4) GAMSSA using in situ SSTs as a reference. In section 3c, we show how The Bureau of Meteorology (BoM) Global Australian Multi- high-resolution products take advantages in intercomparisons. Sensor SST Analysis (GAMSSA) v1 is a daily 0.258 3 0.258 In section 3b, we demonstrate how QC procedures and product starting from 2008 (Zhong and Beggs 2008; Beggs et al. moored buoy observations may affect the reference SST and 2011, 2020). GAMSSA uses SSTs derived from AVHRR, the therefore the assessment of products. The results are summa- Advanced Along Track Scanning Radiometer (AATSR), the rized and discussed in section 4. AMSR2, and in situ SSTs from ships, drifting and moored buoys. Biases in AVHRR and AMSR2 SSTs are adjusted using 2. Datasets and methods drifting buoy SSTs. a. Nine SST products 5) OSTIA 1) DOISST 8 8 The Met Ofﬁce OSTIA v2 is a daily 0.05 3 0.05 SST product The NOAA DOISST v2.1 (Table 1) is a daily 0.258 3 0.258 starting from 2006 (Stark et al. 2007; Donlon et al. 2012; Good product starting September 1981 (Reynolds et al. 2007; Huang et al. 2020). OSTIA includes in situ SSTs from ships, drifting and et al. 2021a). DOISST includes SST observations from ships, moored buoys, satellite SSTs derived from AVHRR, AMSR2, drifting and moored buoys, Argo ﬂoats, and Advanced Very VIIRS, the Sea and Land Surface Temperature Radiometer High Resolution Radiometer (AVHRR) retrieved from NOAA (SLSTR), and the Spinning Enhanced Visible and Infrared series and MetOp-A/-B satellites by U.S. Navy (Huang et al. Imager (SEVIRI). SSTs from drifting and moored buoys 2021a) before November 2021. After November 2021, DOISST and VIIRS nighttime SSTs are used to adjust the biases in other switched to NOAA Advanced Clear Sky Processor for Ocean (ACSPO; Jonasson et al. 2020) satellite SSTs retrieved from satellite-derived SSTs. Biases in satellite SSTs in a 78 grid AVHRR and the Visible Infrared Imager Radiometer Suite are estimated with pairs of in situ SSTs within 25 km. APRIL 2023 HUA N G E T A L . 457 TABLE 1. Daily SST datasets (from January 2016 to January 2022) used in this study (all data were downloaded on 15 Feb 2022). Dataset Version Resolution Input Method Access DOISST v2.0 (1981–2019) 0.258 AVHRR/ACSPO 1 Ship 1 OI https://www.ncei.noaa.gov/data/sea- v2.1 (2016–present) Buoy 1 Argo surface-temperature-optimum- interpolation/v2.1/access/avhrr/ MUR25 MUR v4.2 (2002– 0.258 AVHRR 1 Microwave 1 Multi-Resolution https://podaac-opendap.jpl.nasa.gov/ present) Ship 1 Buoy 1 Argo Variational opendap/allData/ghrsst/data/GDS2/ Analysis (MRVA) L4/GLOB/JPL/MUR25/v4.2 MGDSST (1982–present) 0.258 AVHRR 1 Microwave 1 OI http://www.data.jma.go.jp/gmd/ Ship 1 Buoy goos/data/pub/JMA-product/ mgd_sst_glb_D GAMSSA v1 (2008–present) 0.258 AVHRR 1 AATSR 1 OI https://archive.podaac.earthdata. AMSRE 1 Ship 1 nasa.gov/podaac-ops-cumulus- Buoy 1 ACSPO protected/GAMSSA_28km- ABOM-L4-GLOB-v01 OSTIA v2 (2006–present) 0.058 AVHRR 1 AMSR2 1 OI https://archive.podaac.earthdata. VIIRS 1 SEVIRI 1 nasa.gov/podaac-ops-cumulus- SLSTR 1 Ship 1 Buoy protected/OSTIA-UKMO-L4- GLOB-v2.0 GPB v1 (2014–present) 0.05 Imager 1 AVHRR 1 OI https://archive.podaac.earthdata. VIIRS 1 Ship 1 Buoy nasa.gov/podaac-ops-cumulus- protected/Geo_Polar_Blended_ Night-OSPO-L4-GLOB-v1.0 CCI v2.0 (1981–2019) 0.058 AVHRR 1 ATSR 1 Variational https://dap.ceda.ac.uk/neodc/c3s_ ATSR2 1 Adv. ATSR assimilation (VA) sst/data/ICDR_v2/Analysis/L4/ v2.0;https://dap.ceda.ac.uk/ neodc/esacci/sst/data/CDR_v2/ Analysis/L4/v2.1 CMC v3 (2016–present) 0.18 AVHRR 1 AMSR2 Ship 1 OI https://archive.podaac.earthdata. Buoy nasa.gov/podaac-ops-cumulus- protected/CMC0.1deg-CMC-L4- GLOB-v3.0 GMPE v1 (2009–12) 0.258 GHRSST ensemble-median } ftp://nrt.cmems-du.eu/Core/SST_ v2 (2012–17) SST GLO_SST_L4_NRT_ v3 (2017–present) OBSERVATIONS_010_005/ METOFFICE-GLO-SST-L4- NRT-OBS-GMPE-V3 6) GPB 8) CMC The NOAA Geo-Polar Blended (GPB) v1 is a daily The Canadian Meteorological Centre SST (CMC) v3 is a 0.058 3 0.058 SST product starting from 2014 (Maturi et al. daily 0.18 3 0.18 SST starting from 2016 (Brasnett 1997, 2008; 2017). GPB includes in situ SSTs from ships, drifting and moored Brasnett and Colan 2016). CMC v3 uses in situ SSTs from buoys, and nighttime SSTs derived from AVHRR, VIIRS, the ships and drifting buoys, and AVHRR SSTs from satellites Geostationary Operational Environmental Satellite (GOES) im- NOAA-18 and NOAA-19, MetOp-A and MetOp-B, and ager, and the Japanese Advanced Meteorological Imager AMSR2. Biases in satellite SSTs in 2.58 grid are estimated (JAMI) (Xu and Ignatov 2010). Biases in satellite SSTs are cor- with pairs of in situ SSTs within 25 km. rected by in situ SSTs in a 78 grid basedonpairs of in situ and 9) GMPE satellite SSTs within 25 km. Additionally, the difference between satellite and GPB analysis of the previous day is corrected by an The GMPE is a daily 0.258 3 0.258 product starting from independent NCEP SST product (Thiébaux et al. 2003). 2009 (Martin et al. 2012; Dash et al. 2012; Fiedler et al. 2019). The GMPE selects the median SST from the GHRSST prod- 7) CCI ucts. GMPE v2 (2016) and v3 (2017–20) are used in this study. The European Space Agency (ESA) Climate Change Ini- b. Reference SSTs from in situ buoy and Argo tiative (CCI) SST version 2.0/2.1 is a daily 0.058 3 0.058 SST observations product from 1981 (Merchant et al. 2014, 2019). The CCI in- cludes both AVHRR and Along-Track Scanning Radiometer In this study, the SSTs from drifting buoys and moored (ATSR) series. The biases in satellite SSTs were adjusted by buoys (simply referred as Buoy hereafter) and Argo observa- recalibrating radiances using a reference channel. tions are used as a reference to assess the nine gridded SST 458 J OUR N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L OGY VOLUME 40 products in section 3. The Buoy SSTs are measured at depth QC procedures applied to observations, however, may vary of 0.2–1.0 m (Castro et al. 2012). The temperature measure- among gridded SST products. Therefore, two QC options are used in our assessment: 1) ﬁltering out the outliers deviated ments of Argo ﬂoats and moored buoys above 5 m depth are from the DOISST FG by more than one SST standard devia- usually averaged and taken as SST observations (Roemmich et al. 2015; Huang et al. 2017, 2021a). Buoy SSTs are retrieved tion (SD; Reynolds et al. 2007), and 2) ﬁltering out the out- from the International Comprehensive Ocean–Atmosphere liers deviated from climatological SST (CLM) by 4 times SST Data Set (ICOADS) release 3.0.2 (Liu et al. 2022), and Argo SDs (Huang et al. 2017). SSTs are derived from the Global Data Assembly Centre Intercomparisons are quantiﬁed by globally averaged bias (Bias) and root-mean-square difference (RMSD), or mean (GDAC; Argo 2000). It should be noted that observation difference (DIFF) and SD: densities are very distinct in Buoy and Argo due to observing frequency of Buoy (6 min to 1 h in moored buoy and 1 h in M N drifting buoy) and Argo (10 days) (Fig. S2), although the col- Bias(t) 5 ∑ ∑ [P(x , y , t) 2 O(x , y , t)] 3 cos(y ), (1) i j i j j W i51 j51 located difference between Buoy and Argo SSTs are small (0.038 6 0.038C; Huang et al. 2017). Therefore, their spatial 0:5 M N and time coverages are different, which may impact the inter- RMSD(t) 5 ∑ ∑ [P(x , y , t) 2 O(x , y , t)] 3 cos(y ) , i j i j j comparison results discussed in sections 3b and 3c. i51 j51 Buoy SSTs are ingested into eight out of the nine gridded (2) products in section 2a except for CCI, and Argo SSTs are used in DOISST and MUR25 (Table 1). Therefore, the im- M N pact of independence of Buoy and Argo observations is dis- DIFF(t) 5 ∑ ∑ [P(x , y , t) 2 O(x , y , t)], (3) i j i j MN i51 j51 cussed in section 4. For comparison purposes, the drifting and tropical-moored buoy SSTs from the iQuam project (Xu and 0:5 M N Ignatov 2010) are also used as a reference for the purposes of 2 SD(t) 5 ∑ ∑ [P(x , y , t) 2 O(x , y , t) 2 DIFF(t)] , i j i j MN assessments in section 3f. i51 j51 (4) c. Intercomparison methods To assess the impacts of spatial resolution on the perfor- where P and O represent product and observed SSTs at grid x mance of gridded SST products in reference to Buoy and and y; x, y, and t represent longitude, latitude, and time, re- Argo SSTs, the nine SST products are compared in two reso- spectively; W represents the integrated weighting of cos(y ). lutions from January 2016 to January 2022: (i) the coarsest The Bias and RMSD in Eqs. (1) and (2) are weighted by resolution (0.258) for all nine products, which is, namely, the 8 8 cos(latitude) because they are calculated on 0.25 3 0.25 grid GMPE convention that degrades the four high resolution SST boxes, while DIFF and SD in Eqs. (3) and (4) are not weighted products (OSTIA, GPB, CCI, CMC) to 0.258 resolution, and by cos(latitude) because they are calculated on pointwise loca- (ii) the original (Orig) products’ resolution (0.058–0.258). The tions by the bilinear interpolation from P to O. The uncertain- reason for degrading these SST products in (i) is to eliminate ties of Bias, RMSD, DIFF, and SDs at 95% conﬁdence level their potential advantage of high resolution so that the inter- are quantiﬁed by estimating the effective sampling number ac- comparisons with those in low resolution become fair. cording to lagged autocorrelation coefﬁcients of time series Likewise, the Buoy and Argo SSTs that have passed estab- (Huang et al. 2021b). The Bias and DIFF could be positive or lished QC procedures are used as a reference in two resolu- negative when an SST product is warmer or colder than the tions: 1) the coarsest resolution (0.258), which is derived using reference SST. Therefore, Bias and DIFF could be canceled box average, and 2) the pointwise in situ locations. The reason with each other when they are integrated over the global for using 0.258 resolution is that in situ observations are ﬁrst oceans. In contrast, RMSD and SD are always positive and cannot be canceled with each other when they are integrated processed into superobservations within analysis grid boxes, and that the reference Buoy SSTs will not be overwhelmed by over the global oceans. the moored buoys that provide high-frequency observations. The reason for using pointwise locations is that observations 3. Intercomparisons were actually taken at these in situ locations. When the point- a. GMPE as a reference wise locations of observations are used as a reference SST in this study, the gridded SST products are interpolated to the The eight gridded SST products were compared with pointwise locations using a bilinear interpolation method, GMPE after OSTIA, CCI, GPB, and CMC were box aver- which linearly in both longitude and latitude interpolates the aged to 0.258 3 0.258 grids from January 2016 to January 2022 gridded SSTs surrounding the pointwise observation within the (Fig. 1a). GMPE has frequently been used as a reference to gridbox. Alternatively, we tested an e-fold distance-weighting assess the performance of SST products (Yang et al. 2021; method, in which the gridded SSTs within 0.258 from the point- Huang et al. 2021a; Dash et al. 2012), because it selects the wise location are averaged according to their distance to the median of various SST products and therefore its biases are pointwise location. Our tests indicate that the results using the relatively small (Fiedler et al. 2019). Comparisons in Fig. 1a e-fold method are very close to the bilinear method. show that the globally averaged biases are generally within APRIL 2023 HUA N G E T A L . 459 FIG. 1. (a) Biases and (b) RMSDs in reference to GMPE in DOISST (solid red), MUR25 (dashed blue), MGDSST (solid black), GAMSSA (dotted green), OSTIA (dotted black), GPB (solid light blue), CCI (solid purple), and CMC (dotted orange). The biases and RMSDs are calculated on 0.258 3 0.258 grids. A 15-day running ﬁlter is applied in plotting. 60.158C varying with time. The biases are mostly positive in and CMC (approximately 0.28C), and higher in DOISST, MUR25 and MGDSST, mostly negative in GAMSSA, OSTIA, MUR25, MGDSST, GAMSSA, and CCI (approximately 0.38C), GPB, and CCI, and near zero in DOISST and CMC (Table 2). which is consistent with the SQUAM analysis at https://www. The biases in MGDSST decrease clearly after 2019, whose rea- star.nesdis.noaa.gov/socd/sst/squam/analysis/l4.The RMSDs are sons are not quite clear but may be associated with 1) the selec- generally higher in boreal summer than in boreal winter, which tion of the GMPE among numerous SST products since its may result from the availability of GMPE that shifts toward the biases relative to Buoy SSTs are stable as discussed in section 3b North Pole in boreal summer since the performance of SST (Fig. 3a) and 2) the use of ACSPO data after December 2018. products are generally worse in high latitudes. The reason for The warm biases in MGDSST and MUR25 may be associated thedata availabilityof GMPE is not clear, but maybethat some with their use of SST observations derived from microwaves SST products are not globally covered or ﬁltered out by sea ice (Crewell et al. 1991). The cold biases in GAMSSA, OSTIA, concentrations. However, GMPE itself is biased when it is compared with and GPB may partially be associated with the use of nighttime VIIRS SST and rejecting SST measurements during daytime in in situ Buoy and Argo observations during both day- and low wind speed (Martin et al. 2012). TheRMSDs arebetween nighttime from January 2016 to January 2022 (Fig. 2). GMPE 0.18 and 0.58C(Fig. 1b). The RMSDs are low in OSTIA, GPB, relative to Buoy SSTs has cold biases (20.18C) in the tropical 460 J OUR N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L OGY VOLUME 40 TABLE 2. Averaged biases and RMSDs (8C) in reference to GMPE, Buoy, and Argo SSTs on 0.258 3 0.258 grids from 1 Jan 2016 to 31 Jan 2022 in Figs. 1 and 3, and S1. The 6 values represent the uncertainty at 95% conﬁdence level that is determined by the lagged autocorrelation, effective sampling number, and the standard deviation (SD) (Huang et al. 2021b). GMPE reference Buoy reference Argo reference SST product Bias RMSD Bias RMSD Bias RMSD DOISST v2.1 0.002 6 0.017 0.357 6 0.014 20.018 6 0.013 0.376 6 0.009 20.033 6 0.007 0.346 6 0.002 MUR25 0.087 6 0.010 0.281 6 0.011 0.038 6 0.010 0.531 6 0.015 0.036 6 0.005 0.377 6 0.004 MGDSST 0.051 6 0.029 0.391 6 0.016 0.028 6 0.009 0.650 6 0.047 0.006 6 0.019 0.523 6 0.011 GAMSSA 20.024 6 0.019 0.303 6 0.022 20.071 6 0.011 0.505 6 0.024 20.088 6 0.010 0.480 6 0.005 OSTIA 20.020 6 0.011 0.200 6 0.016 20.045 6 0.011 0.431 6 0.012 20.069 6 0.011 0.370 6 0.042 GPB 20.016 6 0.014 0.203 6 0.041 20.051 6 0.015 0.505 6 0.016 20.066 6 0.011 0.381 6 0.020 CCI 20.025 6 0.013 0.349 6 0.018 20.054 6 0.011 0.608 6 0.036 20.068 6 0.008 0.429 6 0.017 CMC 20.001 6 0.006 0.182 6 0.012 20.052 6 0.009 0.492 6 0.015 20.056 6 0.007 0.380 6 0.002 GMPE }} 20.036 6 0.012 0.454 6 0.012 20.055 6 0.007 0.363 6 0.011 oceans between 158S and 308N, warm biases (0.18 to 0.28C) averaged bias and RMSD are about 20.048 and 0.58C, respec- south of 158S in the Indian Ocean sector and south of 308Sin tively (Table 2). the Paciﬁc–Atlantic sectors, and warm biases (0.28 to 0.48C) in The spatial distribution of biases and RMSDs of GMPE the regions of the Gulf Stream and Kuroshio. The RMSDs relative to Argo SSTs are similar to those relative to Buoy areabove1.08C in the regions of the Gulf Stream and Kuroshio, SSTs (Figs. 2c,d). Exceptions are that the cold biases and 0.68 to 1.08C in the Southern Ocean between 308 and 608S, and RMSDs are slightly lower in the tropical Paciﬁc and Indian approximately 0.28C in the rest of the oceans. The globally Oceans; the warm biases and RMSDs are slightly lower in the FIG. 2. (a) Bias and (b) RMSD of GMPE in reference to Buoy SSTs (8C) on 0.258 3 0.258 grids from 1 Jan 2016 to 31 Jan 2022. (c),(d) As in (a) and (b), but in reference to Argo SST. The biases in (a) and (c) are stippled when they are signiﬁcant at the 95% conﬁdence level, and the areas without observations are shaded with gray. APRIL 2023 HUA N G E T A L . 461 FIG. 3. (a) Biases and (b) RMSDs in reference to Buoy SSTs in DOISST (solid red), MUR25 (dashed blue), MGDSST (solid black), GAMSSA (dotted green), OSTIA (dotted black), GPB (solid light blue), CCI (solid purple), CMC (dotted orange), and GMPE (solid orange). The 8 8 biases and RMSDs are calculated on 0.25 3 0.25 grids. A 15-day running ﬁlter is applied in plotting. regions of the Gulf Stream and Kuroshio and the Southern performance of other SST products. A better reference is Ocean. The differences relative to Buoy and Argo SSTs high-quality in situ observations such as Buoy and Argo SSTs. may result from the differences of spatial and time cover- b. Buoy and Argo as references ages in Buoy and Argo since their observation densities are very distinct (Fig. S2), since the collocated difference The globally averaged biases (Biases) and RMSDs [Eqs. (1) between Buoy and Argo SSTs are small (0.038 6 0.038C; and (2)] relative to ICOADS Buoy (both drifting and moored Huang et al. 2017). Nevertheless, the globally averaged bias buoys) observations in the nine gridded SST products including and RMSD are approximately 20.068 and 0.48C(Table 2), GMPE are calculated on 0.258 3 0.258 grids (Fig. 3a). Figure 3a respectively, which is comparable with the comparisons shows that MGDSST and MUR25 are largely warm biased, against Buoy SSTs. which may be associated with their use of SST observations Our analyses indicate that the magnitude of biases and derived from microwaves (Crewell et al. 1991). In contrast, RMSDs in GMPE relative to in situ SSTs are comparable to DOISST, GAMSSA,OSTIA,GPB,CCI,CMC,and GMPE those in gridded SST products relative to GMPE. Therefore, are cold biased. The cold bias in GMPE is clearly seen. The it is problematic using GMPE as a reference SST to assess the magnitude of the biases is relatively small in DOISST (20.028C; 462 J OUR N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L OGY VOLUME 40 Table 2) and MGDSST (10.038C). The cold biases in GAMSSA, c. Impact of resolution of SST products OSTIA, GPB, and CMC may partially be associated with the bias 1) IN REFERENCE TO BUOY SSTS correction algorithms using nighttime VIIRS SST and rejecting SST measurements during daytime in low wind speed (Martin The Buoy SSTs from ICOADS used in the intercompari- et al. 2012). Figure 3b shows that RMSDs are relatively small in sons in section 3b are box averaged to the daily 0.258 3 0.258 DOISST (0.388C) and OSTIA (0.438C), large in MGDSST and grids before comparison to SST products. This is to evaluate CCI, and in between in MUR25, GAMSSA, GPB, CMC, and analysis accuracy at the grid scale. For highest-resolution vali- GMPE, which is consistent with Huang et al. (2021b).The low dation, it would be necessary to interpolate the analyses to the buoy locations and times before comparison, as in Dash RMSD in DOISST may result partially from the FG QC proce- dure applied to the in situ Buoy (and Argo) observations in both et al. (2012) and Fiedler et al. (2019). DOISST analysis and reference SST, which will be discussed fur- By interpolating gridded SST products to the in situ loca- ther in section 3d. tions of Buoy SSTs, SDs and DIFFs without cos(latitude) In comparison with Argo observations, shown in supplemental weighting [Eqs. (3) and (4)] are calculated because these are materials(Fig. S3a), the biases in the nine gridded products are considered to be point values, rather than grid values repre- similar to those in comparison with Buoy observations (Fig. 3a). sentative of a region that varies with latitude. SDs are close to Exceptions are that the warm biases decrease in MGDSST after the RMSDs without cos(latitude) weighting since DIFFs are 2019 and in MUR25 after 2020. The biases remain small in relatively small. The use of SDs rather than RMSD is for the DOISST (20.038C; Table 2) and MGDSST (10.018C). RMSDs intercomparison purpose as presented in SQUM (Dash et al. (Fig. S3b) are mostly smaller than those relative to Buoy SSTs 2012; Fiedler et al. 2019; https://www.star.nesdis.noaa.gov/ (Fig. 3b), which may be associated with the low observing socd/sst/squam/analysis/l4). Our analyses showed that SDs in frequency of Argo SSTs and will be discussed further in in situ locations are sensitive to the spatial resolution of SST section 3c(2). The maximum RMSD reduces from about products, and therefore the intercomparisons hereafter are 8 8 8 0.8 C(Fig. 3b)to 0.5 –0.6 C(Fig. S3b) in MGDSST and CCI, al- focused on DOISST, OSTIA, GPB, CCI, and CMC with reso- though the RMSD does not reduce much in DOISST. The lutions of 0.258, 0.058, 0.058, 0.058, and 0.108, respectively. The RMSDs are relatively low and close with each other in MUR25, results for MUR25, MGDSST, and GAMSSA are similar to OSTIA, GPB, CCI, CMC, and GMPE, while the RMSDs are DOISST due to their coarse resolution of 0.258. relatively higher in MGSDDT and GAMSSA. Overall, the As a reference, SDs are calculated ﬁrst between gridded RMSDs remain relatively small in DOISST (0.358C) and OSTIA SST products and Buoy observations on 0.258 3 0.258 grids 8 (Fig. 4a). SDs are small in DOISST (0.408C; Table 3) and (0.37 C), being consistent with the comparison with Buoy obser- vations. However, the low RMSDs in DOISST may in part result larger in CCI (0.648C), which is consistent with the assessment from the inclusion of Argo observations (Huang et al. 2021a). It using RMSDs. The high SD in CCI may result from the fact is interesting to note that there is a negative trend of RMSD that CCI is independent from Buoy observations (Table 1). among most of the SST products except for MGDSST and The contrast of SDs in DOISST and CCI is consistent with GAMSSA, which may represent the improvements of the SST that of RMSDs in Table 2. When these SST products on 0.258 3 0.258 grids are inter- analyses, Argo observations, and satellite observations. The main body of this paper in the following sections polated to in situ locations of Buoy observations (Fig. 4b), focuses on the buoy data as the reference data source, the overall SDs decrease in all ﬁve products. Exceptions are whereas the results using Argo ﬂoat data as the refence are that the maximum SDs increase in CCI during the boreal provided in the supplements. We focus on the buoy data for summer of 2016, 2017, and 2019, and increase in CMC in the the following reasons: 1) there are many more (10 times or boreal summer of 2019. The reason for the large seasonal higher; Huang et al. 2017) Buoy than Argo SSTs as indicated peaks of SDs in CCI are not clear. But it might be possible by the observation density shown in the supplemental Fig. S2, that CCI is less reliable in the high latitudes, which is de- and therefore validations against Buoy observations is more tected by more observations during the boreal summer. Par- reliable; 2) the observing frequency of Buoy (6 min to 1 h in ticularly, these observations are independent from CCI. The moored buoy and 1 h in drifting buoy) is much higher than peaks in boreal summer are also visible in other products that of Argo (10 days), which can better resolve the high- due to the same reasons, but the peaks are not as strong as frequency (1 day) SST variability; 3) DOISST ingests Argo those in CCI due to dependence between SST products and while most of other products do not, and therefore the valida- Buoy observations. tions against Argo are unfair to other products. In contrast, On average from January 2016 to January 2022, SDs are the validations against Buoy observations are fair to all prod- small in DOISST (0.388C; Table 3) and OSTIA (0.398C) and ucts except for CCI, and 4) overall, the conclusions are consis- larger in CCI (0.598C), indicating a good performance of tent regardless as to whether Buoy or Argo SSTs are used in DOISST and OSTIA. However, the amplitude of SD decrease the validations. Despite the differences between Buoy and is much smaller in DOISST (about 0.028C) than the other four Argo observations, their SST differences at collocated grid products (about 0.098C), which may indicate an advantage of 8 8 boxes are small (0.03 6 0.03 C) (Huang et al. 2017). There- the higher spatial resolution in OSTIA, GPB, CCI, and CMC. fore, the ﬁgures and tables for validations using Argo as a ref- Therefore, the performance difference among the ﬁve products erence are put to the supplemental materials. becomes small. APRIL 2023 HUA N G E T A L . 463 8 8 FIG. 4. SDs between SST products (Prod) and Buoy SSTs matched with (a) Prod: 0.25 , Buoy: 0.25 ,and QC:FG; (b) Prod: 0.258; Buoy: Pointwise in situ (Situ), and QC: FG; (c) Prod: Orig, Buoy: Situ, and QC: FG; (d) Prod: 0.25, Buoy: Situ, and QC: CLM; and (e) Prod: Orig, Buoy: Situ, and QC: CLM. SDs are calculated without cos(latitude) weighting in DOISST (solid red), OSTIA (solid black), GPB (dotted light blue), CCI (solid purple), and CMC (dotted orange). A 15-day running ﬁlter is applied in plotting. The advantage of high resolution in OSTIA and CMC is with time. The improvement of OSTIA performance may re- clearer when these gridded SST products are interpolated sult from its unique use of SEVIRI and SLSTR whose quality from their original high-resolution grids to the in situ loca- improves with time. tions of Buoy observations (Fig. 4c), which is indeed the case The advantage of high resolution in OSTIA and CMC may in Dash et al. (2012; https://www.star.nesdis.noaa.gov/socd/sst/ be understood by the distance between the latitude–longitude squam/analysis/l4) and Fiedler et al. (2019). SDs become locations of gridded SST products and in situ locations of lower in OSTIA (0.348C; Table 3) and CMC (0.378C) than Buoy observations. Assuming the Buoy observations were DOISST (0.388C). In addition, the SDs in OSTIA decrease randomly distributed within a typical grid box of the SST 464 J OUR N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L OGY VOLUME 40 TABLE 3. SDs (8C) between SST products (Prod) and Buoy SSTs with different matching methods from 1 Jan 2016 to 31 Jan 2022 in Fig. 4. The 6 values represent the uncertainty at 95% conﬁdence level. Globally averaged SDs are calculated without cos(latitude) weighting. Prod: 0.258, Buoy: Prod: 0.258, Buoy: Prod: Orig, Buoy: Prod: 0.258, Buoy: Prod: Orig, Buoy: SST product 0.258, QC: FG Situ, QC: FG Situ, QC: FG Situ, QC: CLM Situ, QC: CLM DOISST v2.1 0.399 6 0.018 0.381 6 0.021 0.381 6 0.021 0.483 6 0.032 0.483 6 0.032 OSTIA 0.481 6 0.025 0.391 6 0.025 0.339 6 0.055 0.477 6 0.024 0.429 6 0.025 GPB 0.536 6 0.021 0.443 6 0.017 0.430 6 0.018 0.504 6 0.012 0.493 6 0.013 CCI 0.643 6 0.041 0.587 6 0.067 0.593 6 0.069 0.573 6 0.026 0.569 6 0.027 CMC 0.523 6 0.023 0.435 6 0.027 0.374 6 0.012 0.520 6 0.029 0.447 6 0.018 products, which should be reasonable in the global oceans OSTIA, GPB, and CMC (Figs. S4a–c; Table S1). The low SD and within a long time period, the averaged distance from the in DOISST may partly result from that DOISST ingests the center of a grid box to a Buoy observation can statistically be Argo SSTs so that they are not independent, which will be approximated by 0.38d, where d is the size of the grid box discussed further in section 4. In addition, there exhibits some interesting features in the SDs: 1) the SDs are overall smaller (https://math.stackexchange.com/questions/15580). Therefore, than those relative to Buoy SSTs, particularly in CCI; the low for the SST products that have a higher resolution as in SDs relative to Argo may partially result from the weaker OSTIA and CMC, the distance from their central grid point diurnal variation in Argo than in Buoy SSTs, since the depth to a Buoy observation is shorter, which may enable the SST of SST measurement is deeper in Argo (above 5 m) than in products to be closer to the Buoy observation and therefore a Buoy (0.2–1.0 m); 2) the SDs do not change much whether smaller SD of the SST differences. these products are interpolated from their 0.25 or original Furthermore, SDs may be associated with the spatial scale resolution to the Argo locations, which may be due to the fact of satellite bias correction. In OSTIA and CMC, the biases of that Argo SSTs are independent from most of these SST satellite SSTs are analyzed on 78 and 2.58 grids, respectively, products; 3) the differences of the SDs among products are according to the matchups of in situ and satellite SSTs within small in comparison with those relative to Buoy SSTs, which 25 km and 1 day. This indicates that the satellite SSTs in these may suggest that the performance of these SST products are two gridded products were in principle adjusted according to close at a longer time scale since Argo SSTs are sampled at the nearby (25 km) in situ SSTs. In contrast, in DOISST, the 10-day cycle; and 4) SDs decrease generally with time, which biases are calculated within 3000 km in latitude, 5000 km in lon- may indicate a general improvement of these SST products gitude, and 15-day data window. The bias correction in a small due to improved observations. scale in OSTIA and CMC may explain why their SDs are lower In comparison with Argo SSTs, DIFFs are low in DOISST when SSTs are interpolated from their original grid to in situ and CMC,higher in OSTIA,GPB,and CCI (Figs. S5a–5c; locations of Buoy SSTs. In GPB, the bias correction procedure Table S2). The low DIFF in DOISST may be associated with is the same as in OSTIA. However, the SST analysis in GPB is the bias correction to the satellite SST within a very large spatial further adjusted by an independent NCEP analysis to prevent a scale as described in section 3c(1). The DIFFs do not change slow drift in its analysis, which may explain why the SD in GPB much whether these SST products are interpolated from 0.258 is not sensitive to how its analysis is interpolated to in situ loca- or their original resolution to in situ Argo locations. These are tions of Buoy SSTs. In CCI, its analysis does not use in situ SSTs consistent with the comparisons in reference to Buoy SSTs. at all, and therefore its SD does not change whether CCI is com- pared with Buoy SSTs in 0.258 or original 0.058 resolution. d. Impact of QC for reference SSTs In contrast to the reductions of SDs, DIFFs do not change The intercomparisons in section 3c use the reference Buoy much whether gridded SST products are interpolated from in and Argo SSTs that have passed the FG QC, which ﬁlter out 0.258 or their original resolutions to in situ Buoy locations the outliers beyond one SST SD from the FG as described in (Figs. 5a–c; Table 4). The robust DIFFs may result from that section 3b. Since the same FG QC was applied in the DOISST the biases in different regions have been cancelled with each analysis, one may argue that the good performance of DOISST other. Overall, DOISST has a lower DIFF. The low DIFF in in the intercomparisons may result from using the same in situ DOISST may result from that the biases in satellite SSTs in data that have passed the FG QC. Indeed, the SST observations DOISST are corrected in very large spatial scales, and there- ﬁltered out by the FG QC in DOISST analysis may not be ﬁl- fore globally averaged DIFF is smaller in DOISST. However, tered out in the other gridded SST analysis systems, and vice the DIFF in DOISST increases after 2020, which may result versa. To clarify whether the performance of DOISST relies on from a constant bias correction of ship observations based on using FG QC applied reference SSTs, the intercomparisons are 2016–19 values (Huang et al. 2021a). repeated in reference to the CLM QC applied Buoy and Argo SSTs. The CLM QC ﬁlters out outliers beyond four SST SDs 2) IN REFERENCE TO ARGO SSTS from SST climatology. Overall, the CLM QC is relatively loose In comparison with the FG QC applied Argo SSTs, SDs and more observations can pass the CLM QC. The CLM QC is are low in DOISST, slightly higher in CCI, and in between in also more independent from SST analysis systems, since the APRIL 2023 HUA N G E T A L . 465 FIG.5.As in Fig. 4, but for DIFFs. SST climatologies are closer than the SST FGs among SST In comparison with SDs against Buoy SSTs that have products. However, it should be noted that the selections of one passed the FG QC, the SDs in reference to the CLM QC SD in FG QC and four SDs in CLM QC are somehow subjec- applied Buoy SSTs increase in all gridded products whether tive and can be different among analysis systems. these SST products are interpolated from 0.258 or their TABLE 4. As in Table 3, but for DIFFs Fig. 5. Prod: 0.258, Buoy: Prod: 0.258, Buoy: Prod: Orig, Buoy: Prod: 0.258, Buoy: Prod: Orig, Buoy: SST product 0.258, QC: FG Situ, QC: FG Situ, QC: FG Situ, QC: CLM Situ, QC: CLM DOISST v2.1 20.025 6 0.012 20.008 6 0.007 20.008 6 0.007 20.011 6 0.011 20.011 6 0.011 OSTIA 20.048 6 0.012 20.043 6 0.010 20.038 6 0.008 20.046 6 0.012 20.043 6 0.009 GPB 20.057 6 0.016 20.055 6 0.012 20.060 6 0.012 20.06 6 0.014 20.064 6 0.014 CCI 20.072 6 0.016 20.067 6 0.012 20.070 6 0.012 20.062 6 0.013 20.066 6 0.013 CMC 20.060 6 0.014 20.056 6 0.011 20.057 6 0.008 20.055 6 0.012 20.063 6 0.010 466 J OUR N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L OGY VOLUME 40 original resolution (Figs. 4d,e; Table 3). The exception is that moored buoys due to use of observations at in situ locations the maximum SDs in CCI decrease clearly during the boreal and times. Since the locations of the moored buoys do not summers of 2016, 2017, and 2019. The reasons for the reduc- change with time, the globally averaged SDs may be biased tion of SDs are not immediately clear but may be associated toward the locations of the moored buoys. with the independence of CCI from in situ observations, be- By removing the moored buoy from the reference Buoy cause the increase of observations in CLM QC validation may SST, the comparisons with drifting-buoy (dBuoy) SSTs in- partially cancel the high SDs in the high latitudes. The time deed show an overall decrease of SDs (Fig. 6 and Table 5) averaged SD against the CLM QC applied Buoy SSTs de- from those in the comparisons with Buoy (both drifting and moored buoys) in Fig. 5 and Table 3. However, the perfor- creases slightly although the SDs during boreal winters do in- crease in CCI. Therefore, it should be cautious in assessment mance among the ﬁve gridded SST products remain similar to of CCI because of its sensitivity to the selection of in situ ob- that in comparison with combined moored- and drifting-buoy servations that have passed different QC criteria. The overall SSTs: SDs are low in DOISST and OSTIA whether FG or increase in SDs in those SST products is intuitive because CLM QCs are applied to dBuoy SSTs and whether these SST more observations are included in reference Buoy SSTs due products are interpolated from 0.258 or original resolution to to a loose CLM QC, which is mostly from moored buoys (by in situ dBuoy locations. The SD is relatively low in CMC 5%–15%) and slightly from drifting buoy (about 2%). Never- when the analysis is interpolated from its original high resolu- theless, our conclusions remain the same: SDs are lower in tion and FG QC is applied to the reference dBuoy SSTs. DOISST and OSTIA when those SST products in 0.258 reso- However, the SDs in comparison with dBuoy SSTs (Table 5) lution are interpolated to Buoy SSTs; and SDs are lower in remain larger than those in comparison with Argo SSTs OSTIA and CMC when those SST products in their original (Table S1), which may result from that the observing frequency resolution are interpolated to Buoy SSTs, which results from of dBuoy (1 h) remain higher than that of Argo (10 days). It is the advantage of high resolution in OSTIA and CMC as ana- interesting to note that the SDs in OSTIA decrease with time lyzed in section 3c. The better performance of OSTIA in both when compared with the Buoy SSTs with FG QC (Fig. 6c). The FG QC and CLM QC may be associated with using more sat- improvement of OSTIA performance may result from its ellite observations (Table 1). The performance of SST prod- unique use of SEVIRI and SLSTR whose quality improves ucts when they are interpolated from their original resolution with time as noted in section 3c(1). is consistent with the SQUAM analyses at https://www.star. Similar to the comparisons with Buoy and Argo SSTs, DIFFs in comparisons with dBuoy SSTs remain low in DOISST nesdis.noaa.gov/socd/sst/squam/analysis/l4. Comparisons also show that DIFFs are lower in DOISST than the other four whether the SST products are interpolated from 0.258 or their SST products whether they are interpolated from 0.258 or original resolution and whether FG QC or CLM QC is applied their original resolution (Figs. 5d,e; Table 4). The low DIFFs to dBuoy SSTs (Fig. 7; Table 6). The low DIFF in DOISST may in DOISST are associated with the large-scale bias correction result from the algorithm of the bias correction to the satellite to the satellite SSTs, and are consistent with the SQUAM SST within a large spatial scale at 3000–5000 km. analysis. f. Impact of iQuam SST In the comparison against the CLM QC applied Argo SSTs (Figs. S4d,e; Table S1), the overall features of SDs are similar The analysis in section 3e (Fig. 4e) shows overall higher to those against the FG QC applied Argo SSTs described in SDs in comparison with those on the SQUAM website (Fig. section 3c(2). Exception is that there is a clear increase of SDs S1; https://www.star.nesdis.noaa.gov/socd/sst/squam/analysis/ in all gridded products in comparison with those using tight l4). Our analyses demonstrate that the low SDs in SQUAM FG QC, since the CLM QC applied Argo observations are result from using the high-quality (QC ﬂag 5; Dash et al. about 1% more than the FG QC applied Argo observations. 2012) drifting and tropical-moored buoy SSTs from iQuam Likewise, the DIFFs are smaller in DOISST whether these (Xu and Ignatov 2010). SST products are interpolated from 0.258 or their original reso- As an example, Fig. 8a shows the SDs of OSTIA relative to lution and whether FG QC or CLM QC is applied to the refer- iQuam and ICOADS buoy SSTs in January 2020. When drift- ence Argo SSTs (Figs. S5d,e; Table S2). These results suggest ing and tropical moored buoy SSTs from iQuam with SQUAM that the intercomparisons are not sensitive to the selections of QC ﬂag 5 are used as reference, the SD is about 0.208Con QC applied to the reference Buoy and Argo SSTs. global average, which is consistent with the SD in SQUAM website. In contrast, when SSTs with SQUAM QC ﬂag 1–5are e. Impact of moored buoys used as reference, the SD increases to 0.38–0.48C, which is con- In the comparisons in sections 3c and 3d, it is noticed that sistent with that in reference to drifting and moored-buoy SSTs the SDs and RMSDs in reference to Buoy SSTs from from ICOADS with CLM QC analyzed in section 3e. ICOADS are mostly higher than those in reference to Argo The increase in SD results from including more low-quality SSTs, particularly in CCI. The difference may result from the observations from iQuam, which increases from about 32 000 higher observing frequency of SSTs from buoys, particularly to about 37 000 day over the global oceans (Fig. 8b). The in- the moored buoys. The high-frequency variances may not be crease in observations mostly happened in the northwest North well resolved by the gridded SST products at daily resolution, Atlantic, northwest North Paciﬁc, and Southern Hemisphere and that the SDs may be overwhelmed by high-frequency oceans between 308 and 508S(Fig. 9a). The inclusion of the APRIL 2023 HUA N G E T A L . 467 FIG. 6. SDs between SST products (Prod) and dBuoy SSTs matched with (a) Prod: 0.258, dBuoy: Situ, and QC: FG; (b) Prod: Orig, dBuoy: Situ, and QC: FG; (c) Prod: 0.258, dBuoy: Situ, and QC: CLM; and (d) Prod: Orig, dBuoy: Situ, and QC: CLM. SDs are calculated without cos(latitude) weighting in DOISST (solid red), OSTIA (solid black), GPB (dotted light blue), CCI (solid purple), and CMC (dotted orange). A 15-day running ﬁlter is applied in plotting. observations in these regions results in a high SD on global aver- relative to buoy SSTs from iQuam with QC ﬂag 1–5 age, because SST analysis is generally less reliable in those re- (Fig. 8a). The reason may be that the high SDs along the gions (refer to Figs. 2b,d). eastern coasts of North America are compensated by the In comparison with the drifting and tropical-moored buoy low SDs along the western coasts. Furthermore, observa- from iQuam, there are many more observations from drifting tions from ICOADS increase when FG QC is applied in and moored buoy from ICOADS whether CLM or FG QC is comparison to that when CLM QC is applied (Fig. 8b). applied (Fig. 8b). The higher number of observations results However, the SD of OSTIA decreases from about 0.358 to from the inclusion of the moored buoy in the subtropical re- 0.258C(Fig. 8a), because the FG QC guarantees that the gions, particularly along the coasts of North America (Fig. 9b). selected observations are close to the analysis although However, the global averaged SD of OSTIA analysis relative more observations along the eastern coasts of North America to buoy SSTs from ICOADS with CLM QC is close to that are included (Fig. 9c). TABLE 5. SDs ( C) between SST products (Prod) and dBuoy SSTs with different matching methods from 1 Jan 2016 to 31 Jan 2022 in Fig. 6.The 6 values represent the uncertainty at 95% conﬁdence level. Globally averaged SDs are calculated without cos(latitude) weighting. Prod: 0.258, dBuoy: Prod: Orig, dBuoy: Prod: 0.258, dBuoy: Prod: Orig, dBuoy: SST product Situ, QC: FG Situ, QC: FG Situ, QC: CLM Situ, QC: CLM DOISST v2.1 0.297 6 0.007 0.297 6 0.007 0.430 6 0.020 0.430 6 0.020 OSTIA 0.343 6 0.027 0.312 6 0.032 0.462 6 0.017 0.430 6 0.020 GPB 0.396 6 0.018 0.388 6 0.019 0.495 6 0.009 0.485 6 0.010 CCI 0.457 6 0.020 0.462 6 0.019 0.552 6 0.016 0.549 6 0.016 CMC 0.370 6 0.015 0.347 6 0.014 0.491 6 0.014 0.436 6 0.015 468 J OUR N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L OGY VOLUME 40 FIG.7.As in Fig. 6, but for DIFFs. 4. Summary and discussion Biases are low in DOISST and MGDSST. Therefore, the good performance of DOISST and OSTIA is seen. The performance of nine gridded SST products has been (iii) By interpolating to the in situ locations of Buoy obser- assessed by comparing with an ensemble SST product vations, the SDs of the four SST products in their origi- (GMPE) and in situ observations from buoys and Argo ﬂoats nal high resolution are lower than those in GMPE during both day- and nighttime using metrics of Bias, RMSD, (0.258) resolution, indicating an advantage of high reso- mean difference, and standard deviation. Our analyses indi- lution in intercomparison. The SDs are low in DOISST cate the following: and OSTIA. However, the SDs are low in OSTIA and (i) Relative to GMPE, RMSDs are low in CMC, OSTIA, CMC in products’ original resolution, which may be asso- and GPB, and Biases are low in DOISST and CMC. ciated with their high resolution in analyses and small However, GMPE may not be a preferable reference, spatial scale in the bias correction to satellite SSTs. The since its bias and RMSD relative to in situ Buoy and DIFFs or Biases are generally low in DOISST, which may Argo observations are comparable with the bias and be associated with its large spatial scale in the bias correc- RMSD of individual SST products relative to the same tion focused on minimizing the mean bias. Buoy and Argo observations. (iv) The relative performance of the SST products remains un- (ii) On GMPE (0.258 3 0.258) grids, RMSDs are low in changed when different QC criteria are applied to the ref- DOISST and OSTIA relative to Buoy observations, and erence Buoy and Argo observations, although the SDs are TABLE 6. As in Table 5, but for DIFFs in Fig. 7. Prod: 0.258, dBuoy: Prod: Orig, dBuoy: Prod: 0.258, dBuoy: Prod: Orig, dBuoy: SST product Situ, QC: FG Situ, QC: FG Situ, QC: CLM Situ, QC: CLM DOISST v2.1 20.011 6 0.012 20.011 6 0.012 20.021 6 0.011 20.021 6 0.011 OSTIA 20.039 6 0.010 20.035 6 0.009 20.047 6 0.013 20.044 6 0.011 GPB 20.048 6 0.015 20.049 6 0.015 20.054 6 0.015 20.056 6 0.015 CCI 20.044 6 0.011 20.047 6 0.011 20.052 6 0.014 20.055 6 0.014 CMC 20.049 6 0.007 20.051 6 0.007 20.057 6 0.012 20.062 6 0.011 APRIL 2023 HUA N G E T A L . 469 FIG. 8. (a) SDs of OSTIA in January 2020 relative to the ICOADS drifting and moored buoy SSTs by FG QC (solid red) and CLM QC (dotted red), and relative to the iQuam drifting and tropical-moored buoy SSTs by SQUAM QC ﬂag 5 (solid green) and ﬂags 1–5 (dotted green). (b) As in (a), but for daily number of observations. smaller when the tight ﬁrst-guess QC rather than the loose (vi) The magnitude of SDs depends on the selection of refer- climatological QC is applied. ence SSTs. When high-quality SSTs from iQuam with (v) The performance of the SST products relative to Argo SQUAM QC ﬂag 5 are used as reference, SDs decrease SSTs is overall similar to that relative to Buoy (combi- clearly because low-quality observations are excluded in nation of drifting and moored buoys) SSTs. However, reference in regions where SST analyses are less reliable. the SDs and RMSDs are smaller than those relative to Our study indicates that the performance assessment may Buoy SSTs, which is likely associated with the longer ob- serving interval of SSTs from Argo ﬂoats (10 days) than depend on whether the gridded products are compared on the buoys (6 min–1 h), as well as the deeper depth of SSTs GMPE resolution or their original resolutions, whether in situ from Argo ﬂoats (above 5 m) than buoys (0.2–1.0 m). Sim- observations are regridded to the GMPE grids or on their in ilarly, the SDs relative to drifting-buoy SSTs are smaller situ locations, whether comparisons are assessed against drift- ing or moored buoys or Argo ﬂoats, whether the bias correc- than those relative Buoy SSTs due to that the observing interval is shorter in moored buoys (6 min–1h) than in tion applied to satellite SSTs is based on localized matchups drifting buoys (1 h). or large-scale SST patterns, and whether reference SSTs are 470 J OUR N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L OGY VOLUME 40 FIG. 9. Differences of observation numbers of buoy SSTs between: (a) iQuam QC ﬂag 1–5 and QC ﬂag 5, (b) ICOADS CLM QC and iQuam QC ﬂag 1–5, and (c) ICOADS FG QC and CLM QC. The differences are calculated in 28 3 28 grids for the purpose of visualization. independent from SST analyses. The clariﬁcation of these the observations in their in situ locations. Particularly, questions has helped understand why the intercomparisons the reference buoy SSTs include the drifting and moored in Huang et al. (2021a,b) are different from those in Dash buoys from ICOADS in Huang et al. (2021a,b) but only in- et al. (2012; https://www.star.nesdis.noaa.gov/socd/sst/squam/ clude drifting and tropical-moored buoys from iQuam with analysis/l4)and Fiedler et al. (2019): Huang et al. (2021a,b) SQUAM QC ﬂag 5 in Dash et al. (2012).The former con- compared SST analyses in the GMPE grids with in situ obser- tains substantially more observations than the latter, which vations in the GMPE grids, while Dash et al. (2012) and Fiedler is mostly located in the regions where SST analyses are less et al. (2019) compared SST analyses in their original grids with reliable. APRIL 2023 HUA N G E T A L . 471 14, 925–937, https://doi.org/10.1175/1520-0426(1997)014,0925: Our analysis indicates that the performance of DOISST AGAOSS.2.0.CO;2. v2.1 is among the best gridded SST products, which is cer- }}, 2008: The impact of satellite retrievals in a global sea- tainly attributed in part to the use of Argo SSTs as demon- surface-temperature analysis. Quart. J. Roy. Meteor. Soc., strated in Huang et al. (2021a). However, the assessment of 134,1745–1760, https://doi.org/10.1002/qj.319. DOISST in reference to Argo is no longer independent, }}, and D. S. Colan, 2016: Assimilating retrievals of sea surface which may force us to rethink a different way to assess the temperature from VIIRS and AMSR2. J. Atmos. Oceanic Tech- performance of SST products. One method is to preserve the nol., 33,361–375, https://doi.org/10.1175/JTECH-D-15-0093.1. 10% of Buoy and/or Argo observations for independent veri- Castro, S. L., G. A. Wick, and W. J. Emery, 2012: Evaluation of ﬁcation without ingesting the analysis system (Huang et al. the relative performance of sea surface temperature measure- 2021a; Reynolds et al. 2007). The selection of the 10% can be ments from different types of drifting and moored buoys us- ing satellite-derived reference products. J. Geophys. Res., a one-time random draw or multitime bootstraps with a data 117, C02029, https://doi.org/10.1029/2011JC007472. separation between ingest (90%) and validation (10%). An- Chin, T. M., J. Vazquez-Cuervo, and E. M. Armstrong, 2017: A other method is to ﬁnd other independent observations such multi-scale high-resolution analysis of global sea surface tem- as conductivity–temperature–depth (CTD), mechanical bathy- perature. Remote Sens. Environ., 200,154–169, https://doi.org/ thermograph (MBT), expendable bathythermograph (XBT) 10.1016/j.rse.2017.07.029. proﬁles, saildrone measurements, thermosalinograph, and ice Cole, J., 2000: Coastal sea surface temperature and coho salmon buoys (Huang et al. 2018, 2021b; Vazquez-Cuervo et al. 2022; production off the north-west United States. Fish. Oceanogr., Moteki 2022; Zhang et al. 2022). However, SSTs from CTD, 9,1–16, https://doi.org/10.1046/j.1365-2419.2000.00118.x. MBT, XBT, ice buoy observations may also be biased due to Crewell, S., E. Ruprecht, and C. Simmer, 1991: Latent heat ﬂux surface contamination and the retrieval accuracy of measure- over the North Atlantic Ocean}A case study. J. Appl. Meteor., 30,1627–1635, https://doi.org/10.1175/1520-0450(1991)030,1627: ment depth (Huang et al. 2018, 2021b), and spatial coverages LHFOTN.2.0.CO;2. of the SSTs from saildrones, thermosalinograph, and ice buoy D’Agata, S., 2022: Ecosystems services at risk. Nat. Climate maybelow. Change, 12,13–14, https://doi.org/10.1038/s41558-021-01256-7. Dash, P., A. Ignatov, Y. Kihai, and J. Sapper, 2010: The SST Acknowledgments. Authors appreciate two anonymous Quality Monitor (SQUAM). J. Atmos. Oceanic Technol., 27, reviewers’ comments that have greatly improve the manu- 1899–1917, https://doi.org/10.1175/2010JTECHO756.1. script. JAC and LC were supported by the NOAA Climate }}, and Coauthors, 2012: Group for High Resolution Sea Surface Program Ofﬁce (NA20OAR4310339). Temperature (GHRSST) analysis ﬁelds inter-comparisons} Part 2: Near real time web-based Level 4 SST Quality Monitor (L4-SQUAM). Deep-Sea Res. II, 77–80,31–43, https://doi.org/ Data availability statement. All gridded SST products are 10.1016/j.dsr2.2012.04.002. available online and listed in Table 1. The ICOADS data are Donlon, C. J., M. Martin, J. Stark, J. Roberts-Jones, E. Fiedler, available at https://www.ncei.noaa.gov/data/marine/nrt and and W. Wimmer, 2012: The Operational Sea Surface Tem- https://www.ncei.noaa.gov/data/marine/nrt. The Argo data were perature and Sea Ice Analysis (OSTIA) system. Remote provided by the Global Data Assembly Centre (GDAC; https:// Sens. Environ., 116,140–158, https://doi.org/10.1016/j.rse.2010. doi.org/10.17882/42182; http://www.seanoe.org/data/00311/42182). 10.017. The iQuam data are available at https://www.star.nesdis.noaa.gov/ Feudale, L., and J. Shukla, 2011: Inﬂuence of sea surface tempera- socd/sst/iquam/data.html. The ACSPO L3S-LEO SST data are ture on the European heat wave of 2003 summer. Part I: An provided by NOAA STAR in experimental mode (10.5067/ observational study. Climate Dyn., 36, 1691–1703, https://doi. GHLPM-3SS28; 10.5067/GHLAM-3SS28). The access date for all org/10.1007/s00382-010-0788-0. Fiedler, E. K., and Coauthors, 2019: Intercomparison of long-term data is 15 February 2022. sea surface temperature analyses using the GHRSST Multi- Product Ensemble (GMPE) system. Remote Sens. Environ., REFERENCES 222,18–33, https://doi.org/10.1016/j.rse.2018.12.015. Good, S., and Coauthors, 2020: The current conﬁguration of the Argo, 2000: Argo ﬂoat data and metadata from Global Data Assem- OSTIA system for operational production of foundation sea bly Centre (Argo GDAC). SEANOE, accessed 1 February 2022, surface temperature and ice concentration analyses. Remote https://doi.org/10.17882/42182. Sens., 12,720, https://doi.org/10.3390/rs12040720. Beggs, H., A. Zhong, G. Warren, O. Alves, G. Brassington, and Hartmann, D. L., 2015: Paciﬁc sea surface temperature and the T. Pugh, 2011: RAMSSA}An operational, high-resolution, winter of 2014. Geophys. Res. Lett., 42, 1894–1902, https://doi. Regional Australian Multi-Sensor Sea surface temperature org/10.1002/2015GL063083. Analysis over the Australian region. Aust. Meteor. Oceanogr. He, J., and B. J. Soden, 2016: The impact of SST biases on projec- J., 61,1–22, https://doi.org/10.22499/2.6101.001. tions of anthropogenic climate change: A greater role for }}, L. Qi, P. Govekar, and C. Grifﬁn, 2020: Ingesting VIIRS SST atmosphere-only models? Geophys. Res. Lett., 43, 7745–7750, into the Bureau of Meteorology’s operational SST analyses. https://doi.org/10.1002/2016GL069803. 21st GHRSST Science Team Meeting,Online, EUMETSAT, Hobday, A. J., and Coauthors, 2016: A hierarchical approach to https://imos.org.au/ﬁleadmin/user_upload/shared/SRS/SST/Beggs_ deﬁning marine heatwaves. Prog. Oceanogr., 141, 227–238, GHRSST-XXI_Extended_Abstract_06Sep2020.pdf. https://doi.org/10.1016/j.pocean.2015.12.014. Brasnett, B., 1997: A global analysis of sea surface temperature Huang, B., and Coauthors, 2017: Extended Reconstructed Sea Sur- for numerical weather prediction. J. Atmos. Oceanic Technol., face Temperature version 5 (ERSSTv5): Upgrades, validations, 472 J OUR N A L O F A T M O S P H E R I C A N D O C E A N I C T E C H N O L OGY VOLUME 40 and intercomparisons. J. Climate, 30, 8179–8205, https://doi.org/ Moteki, Q., 2022: Validation of satellite-based sea surface temper- 10.1175/JCLI-D-16-0836.1. ature products against in situ observations off the western }}, W. Angel, T. Boyer, L. Cheng, G. Chepurin, E. Freeman, coast of Sumatra. Sci. Rep., 12,92, https://doi.org/10.1038/ C. Liu, and H.-M. Zhang, 2018: Evaluating SST analyses with s41598-021-04156-0. independent ocean proﬁle observations. J. Climate, 31,5015– O’Carroll, A. G., and Coauthors, 2019: Observational needs of 5030, https://doi.org/10.1175/JCLI-D-17-0824.1. sea surface temperature. Front. Mar. Sci., 6, 420, https://doi. }}, C. Liu, V. Banzon, E. Freeman, G. Graham, B. Hankins, org/10.3389/fmars.2019.00420. T. Smith, and H.-M. Zhang, 2021a: Improvements of the Daily Reynolds, R. W., T. M. Smith, C. Liu, D. B. Chelton, K. S. Casey, Optimum Interpolation Sea Surface Temperature (DOISST) and M. G. Schlax, 2007: Daily high-resolution blended analy- version 2.1. J. Climate, 34, 2923–2939, https://doi.org/10.1175/ ses for sea surface temperature. J. Climate, 20, 5473–5496, JCLI-D-20-0166.1. https://doi.org/10.1175/2007JCLI1824.1. }}, }}, E. Freeman, G. Graham, T. Smith, and H.-M. Zhang, Roemmich, D., J. Church, J. Gilson, D. Monsellesan, P. Sutton, 2021b: Assessment and intercomparison of NOAA Daily Op- and S. Wijffels, 2015: Unabated planetary warming and its timum Interpolation Sea Surface Temperature (DOISST) ocean structure since 2006. Nat. Climate Change, 5,240–245, version 2.1. J. Climate, 34, 7421–7441, https://doi.org/10.1175/ https://doi.org/10.1038/nclimate2513. JCLI-D-21-0001.1. Sakurai, T., H. Kobayashi, and A. Yamane, 2019: Report from }}, Z. Wang, X. Yin, A. Arguez, G. Graham, C. Liu, T. Smith, JMA for GHRSST-XX. JMA Rep., 9 pp. and H.-M. Zhang, 2021c: Prolonged marine heatwaves in the Shimada, S., T. Ohsawa, T. Kogaki, G. Steinfeld, and D. Heinemann, Arctic: 1982–2020. Geophys. Res. Lett., 48, e2021GL095590, 2015: Effects of sea surface temperature accuracy on offshore https://doi.org/10.1029/2021GL095590. wind resource assessment using a mesoscale model. Wind En- Jonasson, O., I. Gladkova, A. Ignatov, and Y. Kihai, 2020: Pro- ergy, 18,1839–1854, https://doi.org/10.1002/we.1796. gress with development of global gridded super-collated SST Singels, A., and C. N. Bezuidenhout, 1999: The relationship be- products from low Earth orbiting satellites (L3S-LEO) at tween ENSO and rainfall and yield in the South African NOAA. Proc. SPIE., 11420, 1142002, https://doi.org/10.1117/ sugar industry. S. Afr. J. Plant Soil, 16,96–101, https://doi.org/ 12.2551819. 10.1080/02571862.1999.10634854. Karl, T. R., and Coauthors, 2015: Possible artifacts of data biases Stark, J. D., C. J. Donlon, M. J. Martin, and M. E. McCulloch, in the recent global surface warming hiatus. Science, 348, 2007, OSTIA: An operational, high resolution, real time, 1469–1472, https://doi.org/10.1126/science.aaa5632. global sea surface temperature analysis system. OCEANS Kurihara, Y., T. Sakurai, and T. Kuragano, 2006: Global daily sea 2007–Europe, Aberdeen, United Kingdom, IEEE, https://doi. surface temperature analysis using data from satellite micro- org/10.1109/OCEANSE.2007.4302251. wave radiometer, satellite infrared radiometer and in-situ ob- Sun, J., S. Wu, and J. Ao, 2016: Role of the North Paciﬁc sea sur- servations (in Japanese). Wea. Serv. Bull., 73,1–18. face temperature in the East Asian winter monsoon decadal Lima, F. P., and D. S. Wethey, 2012: Three decades of high- variability. Climate Dyn., 46, 3793–3805, https://doi.org/10. resolution coastal sea surface temperatures reveal more than 1007/s00382-015-2805-9. warming. Nat. Commun., 3,704, https://doi.org/10.1038/ Thiébaux, J., E. Rogers, W. Wang, and B. Katz, 2003: A new ncomms1713. high-resolution blended real-time global sea surface tempera- Liu, C., and Coauthors, 2022: Blending TAC and BUFR marine ture analysis. Bull. Amer. Meteor. Soc., 84, 645–656, https:// in situ data for ICOADS near-real-time release 3.0.2. J. At- doi.org/10.1175/BAMS-84-5-645. mos. Oceanic Technol., 39, 1943–1959, https://doi.org/10.1175/ Vazquez-Cuervo, J., S. L. Castro, M. Steele, C. Gentemann, JTECH-D-21-0182.1. J. Gomez-Valdes, and W. Tang, 2022: Comparison of GHRSST Martin, B., and Coauthors, 2012: Group for High Resolution SST analysis in the Arctic Ocean and Alaskan coastal waters us- Sea Surface Temperature (GHRSST) analysis ﬁelds inter- ing saildrones. Remote Sens., 14, 692, https://doi.org/10.3390/ comparisons. Part 1: A GHRSST Multi-Product Ensemble rs14030692. (GMPE). Deep-Sea Res. II, 77–80,21–30, https://doi.org/ Vibhute, A., S. Halder, P. Singh, A. Parekh, J. S. Chowdary, and 10.1016/j.dsr2.2012.04.013. C. Gnanaseelan, 2020: Decadal variability of tropical Indian Maturi, E., A. Harris, J. Mittaz, J. Sapper, G. Wick, X. Zhu, Ocean sea surface temperature and its impact on the Indian P. Dash, and P. Koner, 2017: A new high-resolution sea sur- summer monsoon. Theor. Appl. Climatol., 141, 551–566, face temperature blended analysis. Bull. Amer. Meteor. Soc., https://doi.org/10.1007/s00704-020-03216-1. 98, 1015–1026, https://doi.org/10.1175/BAMS-D-15-00002.1. Xu, F., and A. Ignatov, 2010: Evaluation of in situ sea surface Merchant, C. J., and Coauthors, 2014: Sea surface temperature temperatures for use in the calibration and validation of sat- datasets for climate applications from phase 1 of the Euro- ellite retrievals. J. Geophys. Res., 115, C09022, https://doi.org/ pean Space Agency Climate Change Initiative (SST CCI). 10.1029/2010JC006129. Geosci. Data J., 1,179–191, https://doi.org/10.1002/gdj3.20. Yang, C., and Coauthors, 2021: Sea surface temperature intercom- }}, and Coauthors, 2019: Satellite-based time-series of sea- parison in the framework of the Copernicus Climate Change surface temperature since 1981 for climate applications. Sci. Service (C3S). J. Climate, 34, 5257–5283, https://doi.org/10. Data, 6,223, https://doi.org/10.1038/s41597-019-0236-x. Miller, R. L., and J. DeCampo, 1994: C COAST: A PC-based pro- 1175/JCLI-D-20-0793.1. gram for the analysis of coastal processes using NOAA Coast- Yates, D., R. W. Vervoort, B. Minasny, and A. McBratney, 2016: The history of using rainfall data to improve production in the Watch data. Photogramm. Eng. Remote Sens., 60, 155–160. grain industry in Australia}From Goyder to ENSO. Crop Mohino, E., S. Janicot, and J. Bader, 2011: Sahel rainfall and de- cadal to multi-decadal sea surface temperature variability. Pasture Sci., 67, 467–479, https://doi.org/10.1071/CP15053. Climate Dyn., 37, 419–440, https://doi.org/10.1007/s00382-010- Zhang, C., and Coauthors, 2022: Evaluation of surface conditions 0867-2. from operational forecasts using in situ saildrone observations APRIL 2023 HUA N G E T A L . 473 in the PaciﬁcArctic. Mon. Wea. Rev., 150, 1437–1455, https:// Zhong, A., and H. Beggs, 2008: Operational implementation of doi.org/10.1175/MWR-D-20-0379.1. global Australian multi-sensor sea surface temperature analy- Zhang, L., 2016: The roles of external forcing and natural variabil- sis. Analysis and Prediction Operations Bulletin, No. 77, ity in global warming hiatuses. Climate Dyn., 47,3157–3169, Bureau of Meteorology, 9 pp., http://www.bom.gov.au/ https://doi.org/10.1007/s00382-016-3018-6. australia/charts/bulletins/apob77.pdf.
Journal of Atmospheric and Oceanic Technology – American Meteorological Society
Published: Apr 6, 2023
Access the full text.
Sign up today, get DeepDyve free for 14 days.