Photometric redshifts for Hyper Suprime-Cam Subaru Strategic Program Data Release 1

Photometric redshifts for Hyper Suprime-Cam Subaru Strategic Program Data Release 1 Abstract Photometric redshifts are a key component of many science objectives in the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP). In this paper, we describe and compare the codes used to compute photometric redshifts for HSC-SSP, how we calibrate them, and the typical accuracy we achieve with the HSC five-band photometry (grizy). We introduce a new point estimator based on an improved loss function and demonstrate that it works better than other commonly used estimators. We find that our photo-z’s are most accurate at 0.2 ≲ zphot ≲ 1.5, where we can straddle the 4000 Å break. We achieve σ[Δzphot/(1 + zphot)] ∼ 0.05 and an outlier rate of about 15% for galaxies down to i = 25 within this redshift range. If we limit ourselves to a brighter sample of i < 24, we achieve σ ∼ 0.04 and ∼8% outliers. Our photo-z's should thus enable many science cases for HSC-SSP. We also characterize the accuracy of our redshift probability distribution function (PDF) and discover that some codes over-/underestimate the redshift uncertainties, which has implications for N(z) reconstruction. Our photo-z products for the entire area in Public Data Release 1 are publicly available, and both our catalog products (such as point estimates) and full PDFs can be retrieved from the data release site, ⟨https://hsc-release.mtk.nao.ac.jp/⟩. 1 Introduction In the era of wide and deep imaging surveys, the photometric redshift technique (hereafter photo-z, see Hildebrandt et al. 2010 and references therein) has become compulsory to uncover the large-scale distance and time information of millions (soon billions) of galaxies. While photo-z algorithms and the photometry measurements have improved significantly over the past two decades (Hildebrandt et al. 2008, 2012; Coupon et al. 2009; Dahlen et al. 2013; Bonnett et al. 2016), the challenge of acquiring photo-z estimates accurate enough to meet the requirements of cosmology and galaxy evolution studies continues to motivate the active development of photometry extraction and photo-z algorithms even today. It is now clear that both template-fitting and machine-learning methods are complementary and necessary to compute meaningful photo-z’s. Template-fitting methods (Arnouts et al. 1999; Bolzonella et al. 2000; Feldmann et al. 2006; Brammer et al. 2008; Kotulla et al. 2009) use known galaxy spectral energy distributions (SEDs) and priors (Benitez 2000; Ilbert et al. 2006; Tanaka 2015) to match the observed colors with predicted ones. Such an approach currently represents the only way to provide photo-z estimates in regions of color/magnitude space where no reference redshifts are available (but see also Leistedt & Hogg 2017). Machine-learning methods (Tagliaferri et al. 2003; Collister & Lahav 2004; Lima et al. 2008; Wolf et al. 2009; Carliles et al. 2010; Singal et al. 2011; Brescia et al. 2016) are complementary as they provide efficient photo-z estimates, in terms of speed and precision, but require a training sample that is a fair representation of the galaxy sample of interest, which is often difficult to construct due to missing regions in the multi-color space. Precise photo-z’s are needed to enable the selection of sharp, non-overlapping redshift bins to “slice” the Universe. For example, cosmic shear studies (Kilbinger et al. 2013; Hildebrandt et al. 2017) suffer from galaxies in adjacent redshift bins that dilute the cosmological signal and increase the importance of systematic biases such as the galaxy intrinsic alignments (Heymans et al. 2013). For galaxy evolution studies, it is often important to infer physical properties of galaxies such as stellar mass in addition to redshifts. It is thus crucial to minimize catastrophic photo-z errors that lead to erroneous physical parameters. The accurate characterization of the true underlying redshift distribution of a galaxy sample remains a major challenge in today’s experiments. With samples composed of hundreds of millions of galaxies, systematic biases now largely dominate over statistical errors, and gathering a complete and numerous calibration sample has become increasingly pressing in the context of current and planned large-scale imaging surveys. Recently, significant progress has been made in building fainter spectroscopic redshift (hereafter spec-z) samples, e.g., DEEP2 (Davis et al. 2003; Cooper et al. 2011, 2012; Newman et al. 2013), VVDS (Le Fèvre et al. 2004, 2005, 2013), VUDS (Tasca et al. 2017), and 3D-HST (Skelton et al. 2014; Momcheva et al. 2016). These are complemented by larger but shallower surveys such as VIPERS (Garilli et al. 2014), SDSS (Alam et al. 2015), Wiggle-Z (Drinkwater et al. 2010) and GAMA (Liske et al. 2015). More complete, but with lower redshift resolution, samples are also available from PRIMUS (Coil et al. 2011; Cool et al. 2013), along with many-band photo-z’s from COSMOS (Laigle et al. 2016). In parallel, the community has developed powerful new tools to identify deficiencies in existing spec-z samples (see, e.g., Masters et al. 2015) in order to help focus resources on targeting specific galaxy populations with adequate instruments. Still, additional effort is required to (1) improve photo-z algorithms to fully exploit the information provided by the calibration samples, (2) gather and homogenize heterogeneous datasets, and (3) fill in the under-represented regions of color/magnitude space with reference redshifts in order to calibrate all of the galaxies observed in the deepest photometric surveys. These challenges are faced by all ongoing and future large-scale photometric surveys, such as the Kilo-Degree Survey (KiDS: de Jong et al. 2013), which started in 2011 and whose aim is to map 1500 deg2 in four optical filters (u, g, r, i) at relatively shallow depths. At a similar depth but over a larger area, the Dark Energy Survey (DES, Flaugher 2005) has been surveying 5000 deg2 in five filters (g, r, i, z, Y) in the southern sky since 2013. In the future, Euclid (Laureijs et al. 2011), a space mission to be launched in 2020, will observe 15000 deg2 in one optical (“vis”) and three near-infrared (Y, J, H) filters, complemented by optical multi-wavelength imaging data from the ground. The Large Synoptic Survey Telescope (LSST: Ivezić et al. 2008) will cover 20000 deg2 square degrees in six filters (u, g, r, i, z, y) over a period of ten years, starting from 2022, with significantly deeper imaging data than the projects described above. Here we present the photo-z results from the Hyper-Suprime-Cam Subaru Strategic Program (HSC-SSP: Aihara et al. 2018a, 2018b), a 300-night deep imaging survey dedicated to cosmology and the study of galaxy formation and evolution. The survey consists of three components: a Wide layer (r ∼ 26 at 5 σ for point sources) over 1400 deg2, a Deep layer (r ∼ 27) over 28 deg2, and an UltraDeep layer (r ∼ 28) over 4 deg2. This paper presents the efforts led by the photo-z team in HSC-SSP to develop new photo-z algorithms, gather a state-of-the art reference redshift sample, deal with an unprecedented amount of data, and release our products to the public. The data presented in this study correspond to Public Data Release 1 (PDR1) and the S16A internal data release. In section 2, we describe the procedures used to build a robust training sample and to validate our photo-z estimates. In section 3, we present our photo-z methods. Section 4 defines our adopted performance metrics. In sections 5 and 6, we characterize our photo-z performance. We give an overview of the photo-z products included in the public release in section 7, and finally conclude in section 8. As our previous internal photo-z releases are often used in our science papers, we briefly summarize our previous data products in appendix 1. Unless otherwise stated, all the magnitudes are AB magnitudes. 2 Training, validation, and test samples The HSC-SSP survey footprint has been designed in order to maximize the overlap with other photometric and spectroscopic surveys, while keeping the survey geometry simple. For photo-z purposes, this means we can exploit a large number of public spectroscopic redshifts in our survey fields and use them to calibrate our photo-z’s. This section describes how we construct the training sample, and how we calibrate, validate, and test our photo-z codes (details of the codes can be found in section 3). 2.1 Construction of the training sample We first collect spectroscopic redshifts from the literature: zCOSMOS DR3 (Lilly et al. 2009), UDSz (Bradshaw et al. 2013; McLure et al. 2013), 3D-HST (Skelton et al. 2014; Momcheva et al. 2016), FMOS-COSMOS (Silverman et al. 2015), VVDS (Le Fèvre et al. 2013), VIPERS PDR1 (Garilli et al. 2014), SDSS DR12 (Alam et al. 2015), GAMA DR2 (Liske et al. 2015), WiggleZ DR1 (Drinkwater et al. 2010), DEEP2 DR4 (Davis et al. 2003; Newman et al. 2013), and PRIMUS DR1 (Coil et al. 2011; Cool et al. 2013). As each of these surveys has its own flagging scheme to indicate redshift confidence, we homogenize them for selection of secure redshift. The redshifts and flags are fed to the HSC database and matched with the HSC objects. This public spec-z table (described in detail on the spec-z page at the data release site) is included in PDR1 of HSC-SSP and made available to the community. Our training data include ∼170 k and 37 k high-quality spec-z and g/prism-z, respectively, taken from the matched catalogs described above. We supplement the training data with ∼170 k COSMOS2015 many-band photo-z’s (Laigle et al. 2016), along with a collection of private COSMOS spec-z’s (Mara Salvato, private communication) exclusively used for our photo-z training (these are not included in PDR1). Data are included in our training set if they meet the following quality cuts: Public spec-z data: 0.01 < z < 9 (no stars, quasars, or failures) σz < 0.005(1 + z) (error cut) SDSS/BOSS: $$\rm{\tt {zWarning}} = 0$$ (no apparent issues) DEEP2: $$\rm{\tt {qFlag}} = 4$$ (>99.5% confidence) PRIMUS: $$\rm{\tt {qFlag}} = 4$$ (very confident) VIPERS: $$\rm{\tt {qFlag}} = 3$$–4 (>95% confidence) VVDS: $$\rm{\tt {qFlag}} = 3$$–4 (>95% confidence) GAMA: $$\rm{\tt {qFlag}} \ge 4$$ (very confident) WiggleZ: $$\rm{\tt {qFlag}} \ge 4$$ (very confident) UDSz: $$\rm{\tt {qFlag}} \ge 4$$ (provisional catalog only includes >95% confidence) FMOS-COSMOS: $$\rm{\tt {qFlag}} = 3$$–4, z > 0.01, flag_star is False (>95% confidence with no stars). 3DHST data: flag_star is False (no stars) 0 < z < 9 (no stars, quasars, or redshift failures) max (z82 − z50, z50 − z18) < 0.05(1 + z) (1 σ redshift dispersion <5%) max (z97.5 − z50, z50 − z2.5) < 0.1(1 + z) (2 σ redshift dispersion <10%). COSMOS data: Spec-z: $$3 \le \rm{\tt {qFlag}} < 6$$ (>99% confidence) 0 < z < 7 (no stars or quasars) For objects with repeat observations, σz < 0.005(1 + ⟨z⟩) (redshifts agree to within 0.5%). Photo-z: flag_capak is False (no bad photometry) $$\rm{\tt {type}} = 0$$ (only galaxies) χ2(gal) < χ2(star) and χ2(gal)/Nbands < 5 (fits are reasonable and better than stellar alternatives) z~secondary <0 (no secondary peaks) log M* > 7.5 (stellar mass recovery successful) 0 < z < 9 (no stars, quasars, or X-ray detected sources) max (z84 − z50, z50 − z16) < 0.05(1 + z) (1 σ redshift dispersion <5%). Objects are subsequently matched directly to a set of target UltraDeep/Deep catalogs selected using the following criteria: detect_is_primary is True (no duplicates) [grizy]cmodel_flux_flags is False [grizy]flags_pixel_edge is False [grizy]flags_pixel_interpolated_center is False [grizy]flags_pixel_saturated_center is False [grizy]flags_pixel_cr_center is False [grizy]flags_pixel_bad is False [grizy]centroid_sdss_flags is False. These are designed to maximize completeness while removing objects with unreliable photometry. Objects are iteratively matched to this modified catalog within 1″ at (1) UltraDeep, (2) Deep, and (3) Wide depths in order to take advantage of higher signal-to-noise (S/N) data when available, while avoiding possible duplicates. The following quantities are then selected and/or computed: Identifiers: ID, (ra,dec), and (tract,patch) coordinates. Fluxes: PSF fluxes, cmodel fluxes, cmodel_exp fluxes, cmodel_dev fluxes, and PSF-matched aperture fluxes with target 1$${^{\prime\prime}_{.}}$$1 PSF and 1$${^{\prime\prime}_{.}}$$5 apertures taken from the afterburner run (afterburner fluxes; Aihara et al. 2018b). Shapes: sdss_shape parameters. Miscellaneous: merge measurement flags, attenuation estimates (a_[grizy]), and extendedness measurements. Redshift: redshift, 1 σ error, parent survey (SDSS, etc.), and redshift type (spectroscopic, g/prism, or many-band photometric). Depth: flag for UltraDeep, Deep, and Wide photometry. Emulated errors: emulated Wide-depth photometric errors. These are relevant for objects from the Deep and UltraDeep layers that have smaller photometric uncertainties than Wide due to the deeper depths. They are computed independently for each flux type (PSF, cmodel, etc.) by assigning S/N values from a grizy nearest-neighbor search to a Wide catalog of ∼500 k objects selected to mimic the overall survey sample. Weights: Color/magnitude weights are computed using a generalization of the Lima et al. (2008) nearest-neighbor approach. The color/magnitude distribution of the training sample is different from that of the target sample from the Wide layer. The weights are computed so that the training sample can reproduce the color/magnitude distribution of the target galaxies. See J. Speagle et al. (in preparation) for more details. The underlying and reweighted magnitude and redshift distributions of our training sample are shown in figures 1 and 2, respectively. Fig. 1. View largeDownload slide Original (dashed) and reweighted (solid) normalized number densities of our training sample as a function of grizy (left-to-right) magnitude (top) and error (bottom). Note that we use asinh magnitudes (i.e., Luptitudes) here. Our color/magnitude weights are able to effectively correct for biases in our original training sample to better mimic the HSC-SSP Wide data. Fig. 1. View largeDownload slide Original (dashed) and reweighted (solid) normalized number densities of our training sample as a function of grizy (left-to-right) magnitude (top) and error (bottom). Note that we use asinh magnitudes (i.e., Luptitudes) here. Our color/magnitude weights are able to effectively correct for biases in our original training sample to better mimic the HSC-SSP Wide data. Fig. 2. View largeDownload slide Re-weighted, normalized redshift number density for our training sample. The full distribution is shown in solid black while the spec-z, g/prism-z, and many-band photo-z components are shown in solid red, purple, and blue, respectively. The dashed lines show these same components renormalized to the full sample in order to better highlight their differences. We can see that most of the substructure in the redshift distribution of our training sample comes from the many-band COSMOS photo-z’s, which also contribute almost all of our high-z sources. Fig. 2. View largeDownload slide Re-weighted, normalized redshift number density for our training sample. The full distribution is shown in solid black while the spec-z, g/prism-z, and many-band photo-z components are shown in solid red, purple, and blue, respectively. The dashed lines show these same components renormalized to the full sample in order to better highlight their differences. We can see that most of the substructure in the redshift distribution of our training sample comes from the many-band COSMOS photo-z’s, which also contribute almost all of our high-z sources. As described here, our training sample consists of various redshift measurements (spec, g/prism, and many-band photo-z’s). We use all of them as the “truth” throughout the paper, but some of the redshifts (especially the many-band photo-z’s) may be erroneous. We thus urge caution when interpreting the absolute numbers in our adopted metrics. We refer to these “true” redshifts as reference redshifts (zref) throughout the paper. 2.2 Training and validation procedures The training sample is split into k = 5 randomized “folds.” Because each fold has a relatively large number of objects (∼75k), most of us employ a simple hold-out validation to train and validate our photo-z methods. To be specific, we use folds k = 1–3 to train our codes and k = 4 to validate them, with the last fold (k = 5) reserved for testing (see below). The only exception here is FRANKEN-Z, which uses cross-validation (i.e., it used five rotating folds for training and validation). Throughout the paper, all statistics are computed using the color/magnitude weights described in the previous section. 2.3 Test samples We reserve a test sample from the training sample in order to evaluate the performance of our codes. Most of us use the subsample of the training sample described in the previous section (the fifth fold). For FRANKEN-Z, we use one of the five-fold cross-validation runs. We use the test sample to evaluate our performance at the UltraDeep depth. This is reasonable because a significant fraction of the objects in the test sample come from UltraDeep COSMOS, especially at faint magnitudes. For the Wide-depth performance evaluation, we stack a subsample of the COSMOS UltraDeep data to the Wide depth in all the bands. We have computed the emulated Wide-depth photometric uncertainties as described earlier, but they turn out to be problematic in a few cases. Some of our codes use multiple photometry techniques (e.g., EPHOR uses exponential and de Vaucouleur fluxes from CModel), but because the measurements are done using the same pixels, these measurements are strongly correlated. The random flux perturbation is no longer valid and we find that the resultant photo-z’s have weird features. We thus resort to the COSMOS Wide-depth stacks. Thanks to the large number of visits available in the field, we could generate stacks with three different seeing FWHMs (0$${^{\prime\prime}_{.}}$$5, 0$${^{\prime\prime}_{.}}$$7, and 1$${^{\prime\prime}_{.}}$$0), which we will later use to evaluate the seeing dependence. We call these stacks the Wide-depth “best,” “median,” and “worst” seeing stacks. Because we use only a small subsample of the UltraDeep COSMOS data (typically 1/10 of all the visits), it is reasonable to assume that the photometry is quasi-independent of the training sample. But, again, we urge caution when interpreting the absolute numbers. We note that the Wide-depth stacks have the same N(z) distribution as the training sample, leading to some drawbacks, as we will discuss in subsection 6.1. The COSMOS Wide-depth stacks are included in the public data release (Aihara et al. 2018b) and can be exploited by the community. We note that the current Wide-depth stacks have a known issue that the i band in the median seeing stack is slightly shallower than the Wide-depth (15 min in total, as opposed to the nominal exposure of 20 min). But, for the purpose of the photo-z analyses in this paper, we do not suffer significantly from this issue because we limit ourselves to relatively bright magnitudes of i < 25. Also, the slightly shallower depth in only one of the five bands does not have a major impact on the overall photo-z performance. 3 Methods As we reviewed in the introduction section, each photo-z technique has pros and cons. For HSC-SSP, we use all the template-fitting, empirical-fitting, and machine-learning techniques to cover the wide range of scientific applications. We describe each of our codes in this section. 3.1 DEmP The Direct Empirical Photometric code (DEmP: Hsieh & Yee 2014) is the successor of the empirical quadratic polynomial photometric redshift fitting code (Hsieh et al. 2005) applied to the Red-Sequence Cluster Survey data. It is designed to minimize the major issues of conventional empirical-fitting methods, e.g., how to choose a proper form for the fitting functions, and biased results due to the population distribution of the training set, by introducing two techniques: regional polynomial fitting and a uniformly weighted training set. The former performs a fit for each input galaxy using a subset of the training set galaxies with photometry and colors closest to those of the input galaxy, and the latter resamples the training set to produce a flat population distribution. However, we find that using a uniformly weighted training set does not improve the overall photo-z quality. This is because the number density of this training set is sufficiently high thanks to the many-band photo-z’s from COSMOS; the subset of the training set used in the regional polynomial fitting consists of galaxies with very similar magnitudes and colors, which reduces the bias caused by the population distribution of the training set. Therefore, we use only the regional polynomial fitting to derive the HSC photo-z's. The probability distribution of photo-z for each galaxy is generated using Monte Carlo techniques and the bootstrapping method. We use Monte Carlo techniques to generate 500 datasets based on the photometry and uncertainties of the input galaxies to account for the effects due to photometric uncertainties. We then bootstrap the training set for each input galaxy 500 times for each of the Monte Carlo generated datasets, to estimate the sampling effect in the training set. More details are described in Hsieh and Lee (2014). We use the PSF-matched aperture photometry (also known as the afterburner photometry; Aihara et al. 2018b) to derive photo-z's for all the primary objects even with only one-band detection. 3.2 Ephor Extended Photometric Redshift (EPHOR) is a publicly available, neural network photo-z code.1 We use a feedforward neural network that has an input layer (x0), a series of hidden layers (xi for i = 1, …, n), and an output layer (y). Variables with bold typeface are horizontal vectors. We feed the neural network with two model fluxes: the de Vaucouleur flux and the exponential flux in each band. These are derived as part of the CModel photometry (Bosch et al. 2018). The fluxes fk are normalized before being fed to the neural network:   \begin{eqnarray} (\mathbf {x}_0)_k &=& \mathop {\mathrm{arsinh}}\left( \frac{f_k - \mu _k}{\sigma _k} \right) , \end{eqnarray} (1)where μk is the median of fk over the training dataset (training as opposed to validation and test), and σk is the interquartile range, non-normalized, of the training dataset. The function arsinh is applied so that unusually large fluxes will not ruin the neural network. The hidden layers employ softplus [$$\mathop {\mathrm{softplus}}(x) = \ln (1 + \mathrm{e}^x)$$] as the activation function:   \begin{eqnarray} \mathbf {x}_i &=& \mathop {\mathrm{softplus}}(\mathbf {x}_{i-1} W_i + \mathbf {b}_i) \ \ \ \ \mbox{for $i = 1,\ldots ,n$} , \end{eqnarray} (2)where Wi is a weight matrix and bi is a bias vector, both of which are determined in the training. The $$\mathop {\mathrm{softplus}}$$ activation function is applied to the argument vector elementwise. The neural network performs slightly better with softplus than with the rectifier f(x) = max (0, x). The output layer is softmax: y = σ(xn), or   \begin{eqnarray} y_k &=& [ \sigma (\mathbf {x}_n)]_k = \frac{\mathrm{e}^{(\mathbf {x}_n)_{k}}}{\sum _{\ell } \mathrm{e}^{(\mathbf {x}_n)_{\ell }}} . \end{eqnarray} (3)We split the range of redshifts at equal intervals z0 < z1 < ⋅⋅⋅ < zd, and equate yk with the probability of the redshift being within the kth bin [zk − 1, zk). We train the neural network by means of ADAM (Kingma & Ba 2014) so that the cross entropy defined below is minimized:   \begin{eqnarray} H &=& \left\langle -\sum _{k=1}^{d} y^{\prime }_k \ln y_k \right\rangle , \end{eqnarray} (4)where the average is taken from the training dataset, and y΄ = (0⋅⋅⋅0 1 0⋅⋅⋅0) is a one-hot vector for a sample:   \begin{eqnarray} y^{\prime }_k &=& \left\lbrace \begin{array}{ll}1 & \mbox{if the sample's redshift is in $[z_{k-1}, z_{k})$} , \\ 0 & \mbox{otherwise} . \end{array}\right. \end{eqnarray} (5)The default setup of EPHOR is to use the two model fluxes in each filter. However, we also run the code using the PSF-matched aperture photometry (one flux in each band), and we refer to the photo-z’s as EPHOR_AB, where AB stands for afterburner. 3.3 FRANKEN-Z Flexible Regression over Associated Neighbors with Kernel dEnsity estimatioN for Redshifts (FRANKEN-Z) is a hybrid approach that combines the data-driven nature of machine learning with the statistical rigor of posterior-driven (i.e., template-fitting) approaches. Using machine learning, FRANKEN-Z attempts to approximate the “flux projection” from a set of unknown target objects to a corresponding set of training objects in the presence of observational errors within both datasets. The corresponding mapping to redshift is then computed by stacking each training object’s posterior-weighted redshift kernel density estimate. This constitutes a generalization of typical template-fitting approaches to the machine-learning regime. For HSC-SSP PDR1, we approximated the associated flux projection using a collection of an object’s nearest neighbors in magnitude space. We incorporated observational errors by selecting object neighbors to be the union of the 10 nearest neighbors in magnitude space computed using the PSF-matched photometry over 25 Monte Carlo realizations. The log-likelihoods for each object i given training object j were then computed using the associated fluxes via   \begin{equation} -2\ln P(i|j) = \sum _{b} \frac{(F_{i,b}-F_{j,b})^2}{\sigma ^2_{i,b}+\sigma ^2_{j,b}} - n(i,j), \end{equation} (6)where the sum is taken over all bands indexed by b, and n(i, j) is the number of bands where both i and j are observed. Because our nearest-neighbor search is in flux rather than redshift, our results are (somewhat) more robust to domain mismatches between the training/target datasets. We thus assume our prior is uniform over our training data such that our posterior is directly proportional to our likelihood. The redshift PDF P(z|i) then constitutes a posterior-weighted sum,   \begin{eqnarray} P(z|i) &=& \sum _j P(z,j|i)\nonumber\\ &=& \sum _j P(z|j) P(j|i) \propto \sum _j P(z|j) P(i|j) , \end{eqnarray} (7)where P(j|i) is the posterior and P(i|j) is again the likelihood. We note that the full code is still under active development and is more flexible than the early version utilized here.2 See J. Speagle et al. (in preparation) for additional details. 3.4 MLZ SOMz is a part of the public photo-z code, MLZ, which enables us to estimate photometric redshift with a self-organizing map (SOM). The SOM algorithm itself is an unsupervised machine-learning method and is widely used to classify a given dataset into small segments with similar properties. For photo-z measurements, we first apply the SOM to the training set and assign a redshift to each segment by computing the mean redshift of the galaxies in that segment. Then, we find the closest segment for every photometric object to assign a redshift. Monte Carlo and bootstrap resampling enables us to produce the probability distribution of every galaxy. We describe each step in more detail. First, we prepare the random map of Npix defined on a two-dimensional sphere, where “pixel” is defined by Healpix pixelization with $$N_{\rm pix}=12\times N_{\rm side}^2$$. The pth pixel has a vector wpi describing the object properties, e.g., five-band magnitudes, where subscript p runs from 1 to Npix, and i = 1, 2, …, Natt, with Natt being the number of properties to characterize objects, i.e., the kth galaxy has data vector $${\boldsymbol x}_k = \lbrace x_{1k}, x_{2k}, \ldots , x_{N_{\rm att} k} \rbrace$$. Here we utilize five band magnitudes of CModel photometry, and ten colors derived from those magnitudes. During the optimization, we find that colors from afterburner photometry, in addition to CModel magnitudes and colors, slightly improve the photo-z performance. Therefore, we characterize objects with 5+10+10 attributes with their measurement errors. Not all the attributes are independent, and there are covariances between them. We ignore the covariances for now and leave it to our future work to evaluate their effects on photo-z’s. As an initial condition of the map, we set the vector value in each pixel to be randomly drawn from the data vector. The Euclidean distance between a given galaxy and a pixel is defined as   \begin{equation} d(p, k) = \sqrt{\sum _i \frac{(w_{\rm pi}-x_{ik})^2}{\sigma _{ik}^2}}. \end{equation} (8)Then we look for the nearest pixel for the given galaxy. For the nearest pixel of the kth galaxy $$\hat{p}(k)$$, $$d(\hat{p},k) \le d(p,k)$$ holds for any p. The weight vectors of the nearest pixel and the vicinity of the nearest pixels are iteratively updated:   \begin{equation} {\boldsymbol{w}}_p(t+1) = {\boldsymbol{w}}_p(t) + \alpha (t) \exp \left[-\frac{ {\boldsymbol{\gamma }}^2(p,\hat{p})}{2\sigma ^2(t)} \right], \end{equation} (9)where $$\gamma (p, \hat{p})$$ is the angular distance between pixel p and the nearest pixel $$\hat{p}$$, and α, σ are monotonically decreasing functions of time t. The time t increments after we use one galaxy. After the pixels are updated using all galaxies, the same processes are iteratively applied except for setting the initial map to be random. We iterate this for Nite times. In order to obtain a reliable redshift probability distribution function, P(z), we make a perturbed catalog using both bootstrap resampling and Monte Carlo methods. For the latter, we perturb all the magnitudes and colors according to their measurement errors. As a result, we have Nboot × NMC samples to derive our final P(z). As described in subsection 2.2, we optimize the hyper-parameters using folds 1–3 and evaluate the performance with fold 4. We note that the optimization is performed in terms of minimizing σconv instead of the loss function introduced in subsection 4.1. That might partly be the reason why MLZ performed worse than the other machine-learning codes, as we discuss later. The hyper-parameters include Npix, Natt, Nite, Nboot, and NMC. Given the reasonable timescale to compute a large number of objects, we find that the optimal hyper-parameter set is Npix = 16, Natt = 5 + 10 + 10 = 25, Nite = 200, Nboot = 24, and NMC = 16. Except for Natt, increases in those parameters do not significantly improve our results. 3.5 NNPZ Nearest Neighbors P(z) (NNPZ) redshifts are computed following the method introduced by Cunha et al. (2009). The principle of the method is explained in their subsection 2.2, and can be summarized as finding the nearest neighbors around an unknown object in the Euclidian color/magnitude space from a reference sample and using the reference redshift histogram as the PDF. There are, however, a number of differences between the original method and the one applied here: i, g − r, r − i, i − z, z − y color/magnitude attributes (CModel photometry) are used. The reference sample is the weighted training sample (folds 1–3) as described in section 2. The neighbors are weighted according to the inverse Euclidean distance in the color/magnitude space. To avoid giving too much weight to a neighbor with low signal-to-noise photometry that accidentally lies very close to the target object, neighbors with large photometric errors are down-weighted. To do so, we first compute a photometric-error estimate as the sum of the photometric errors in all bands from both the unknown and neighbor objects, and we take the inverse of the photometric-error estimate as the weight. The final weight for each neighbor is the product of the reference, distance, and photometric-error weights. The final object P(z) is thus the weighted histogram of the neighbors. We also record the neighbor redshifts and weights in additional output tables. We note that the choice of the maximum number of neighbors, 50 here, has little impact owing to the weighting scheme in color/magnitude space. We do not produce a P(z) when the CModel measurement has failed in any of the bands. 3.6 Mizuki Finally, we use a template-fitting code Mizuki (Tanaka 2015). This code differs from classical template-fitting codes in a few respects. It uses a set of templates generated with the Bruzual and Charlot (2003) stellar population synthesis code assuming a Chabrier (2003) IMF and Calzetti et al. (2000) dust attenuation curve. Emission lines are added to the templates assuming solar metallicity (Inoue 2011). There are pros and cons in using stellar population synthesis models. One disadvantage of using theoretical templates is that they deliver less accurate photometric redshifts than empirical templates because empirical templates often fit the observed SEDs of galaxies better. However, we correct for this template mismatch by applying a template error function (Brammer et al. 2008), which comes in two terms, both as functions of rest-frame wavelength. One is a systematic flux correction applied to the templates to reduce the mismatch, and the other is template flux uncertainty to properly weight (un)reliable parts of SEDs. This template error function can be derived from the data by comparing the best-fit model fluxes and the observed fluxes of objects. We use the training sample (folds 1–3) to generate the template error function. A big advantage of using theoretical templates is that we know the physical properties of galaxies such as SFR and stellar mass for each template. We apply a set of Bayesian priors on the physical properties and let the priors depend on redshift. Refer to Tanaka (2015) for details of the priors, but they are all observationally motivated. What these priors effectively do is (1) keep the template parameters within realistic ranges to reduce degeneracy in the multi-color space, and also (2) let templates evolve with redshift in an observationally motivated way. Both template error functions and the physical priors improve photometric redshifts. An improvement to the original code is that the N(z|mag) prior is extended to multi-color space and now uses N(z|g − i, i − y, i). We make grids in the two-color magnitude space and pre-compute N(z) in each grid using the training sample (folds 1–3). There are some redshift spikes in the COSMOS field and we apply Gaussian smoothing with σz = 0.05 in each grid to largely smear out the COSMOS-specific features. In addition to redshifts, we compute stellar mass, SFR, and extinction fully marginalized over all the other parameters, which can be useful for galaxy science. Appendix 2 compares stellar mass and SFRs from the code against an external multi-wavelength survey. In addition to galaxy templates, we also include quasi-stellar object (QSO) and active galactic nuclei (AGN) templates and stellar templates. The QSO/AGN templates are generated by combining the type-1 QSO spectrum from Polletta et al. (2007) and young galaxy templates from Bruzual and Charlot (2003) assuming τ = 1 Gyr, age < 2 Gyr, and 0 < τV < 2, where τ is an exponential decay timescale of star formation history, age is time since the onset of star formation, and τV is the optical depth (attenuation) in the V band. The relative fractions of the QSO and galaxy components are 0.5:1, 1:1, 2:1, and 4:1. These hybrid templates are similar to those presented in Salvato et al. (2009). For the stellar templates, we use the BaSeL 3.1 stellar library (Westera et al. 2002). These QSO and stellar templates are used to give relative probabilities of objects being galaxy, QSO, or star. At this point, this functionality of the code is still preliminary, and for simplicity, we use stellar and QSO templates for compact sources (we use the standard extendedness parameter from the pipeline down to i ∼ 24 to identify compact objects, and all the fainter objects are assumed to be extended; see Aihara et al. 2018b for details). Only the galaxy templates are used for extended sources. One important caveat in this release is that the code is trained using an old version of the training sample with erroneous weights (the one described in subsection 2.1 but without the centroid_sdss flag cut). We unfortunately did not have time to retrain the code with the new training sample. This might be part of the reason why the code performs worse than the other codes. 4 Metrics and their definitions 4.1 Metrics to characterize photo-z There are a few standard quantities that are used to characterize photo-z accuracy. However, as their definitions are not always the same in the literature, we explicitly define them here for this paper. We also introduce new quantities. Bias: Photo-z’s may systematically be off from spectroscopic redshifts and we call this systematic offset bias. We compute a systematic bias in Δz = (zphot − zref)/(1 + zref) by applying the biweight statistics (Beers et al. 1990). The biweight is a robust statistical method to estimate the center and dispersion of a data sample by applying a weight function to down-weight outliers, which we often have in photo-z’s. We iteratively apply 3 σ clipping three times to further reduce outliers. Dispersion: In the literature, dispersion is often computed as   \begin{equation} \sigma _{\rm conv}=1.48\times {\rm MAD}(\Delta z), \end{equation} (10)where MAD is the median absolute deviation. Note that this definition does not account for the systematic bias. In addition to this conventional definition, we also measure the dispersion by accounting for the bias using the biweight statistics. We iteratively apply 3 σ clipping as done for bias to measure the dispersion around the central value. We denote the conventional dispersion and the biweight dispersion as σconv and σ, respectively. Outlier rate: The conventional definition is   \begin{equation} f_{\rm outlier,conv}=\frac{N\left(|\Delta z|>0.15\right)}{N_{\rm total}}, \end{equation} (11)where outliers are defined as |Δz| > 0.15. Again, this definition does not account for the systematic bias. The threshold of 0.15 is an arbitrary value but is probably reasonable for photo-z’s with several bands. It is clearly too large for those with many bands. Together with this conventional one, we also define outliers as those 2 σ away from the central value (these σ and center are from biweight; see above). This 2 σ is an arbitrary choice, but it is motivated to match reasonably well with the conventional one for photo-z’s in several bands. We will denote the σ-based outlier fraction as foutlier and the conventional one as foutlier, conv. Loss function: It can be cumbersome to use multiple indicators to characterize the photo-z accuracy. Here we define a simple loss function to remedy the complexity and help us capture the photo-z accuracy with a single number. We define a loss function as   \begin{equation} L(\Delta z)=1-\frac{1}{1+\left(\frac{\Delta z}{\gamma }\right)^2}. \end{equation} (12)This is an “inverted” Lorentz function. The loss is zero when Δz = 0 and continuously increases with larger Δz. Thus, this can be considered as a continuous form of the outlier rate defined above. The loss also increases with the photo-z bias, because a systematic bias means non-zero Δz for most objects. The loss also increases with dispersion because a larger dispersion means larger Δz. Therefore, it effectively combines the three popular metrics into a single number. In order to maintain rough consistency with the conventional outlier definition, we adopt γ = 0.15. 4.2 Optimal point estimates and photo-z risk parameter Our photo-z methods do not output a point redshift directly, but instead infer a redshift PDF, P(z). We want to use the full P(z) for science, but it is often useful to reduce the PDF to a point estimate, zphot. There are several ways to do it; the mean, median, or mode of P(z), for example. To obtain the “best” point estimate, however, we take the minimum risk strategy—we define a “risk” parameter as a function of redshift and choose the point where the risk is minimized as the best point estimate. The loss function L(Δz) defined above is a function of zphot and zref, and can be viewed as a loss arising from zphot being different from zref. The expected amount of loss for a point estimate zphot can be estimated as   \begin{eqnarray} R(z_{\mathrm{phot}}) &=& \int \mathrm{d}z P(z) L\left( \frac{z_{\rm phot}-z}{1+z}\right) . \end{eqnarray} (13)The integral R(zphot) depends only on zphot and represents the expected loss for a given choice of zphot as the point estimate. We employ R(zphot) as the “risk” function. The risk R(zphot) can be roughly interpreted as the probability of the inferred redshift zphot being an outlier: the loss L(Δz) is approximately 0 if the guess zphot is close to the true answer zref, and it is approximately 1 if the guess zphot differs largely from the true answer zref. As mentioned above, we take the minimum risk strategy to choose a point estimate zphot at which the risk R(zphot) is minimum, which we call the best point estimate zbest:   \begin{eqnarray} z_{\mathrm{best}} = \mathop {\mathrm{argmin}}[R(z_{\mathrm{phot}})] . \end{eqnarray} (14)This minimal point has no closed-form solution and must be searched for numerically. In addition to zbest, we also compute zmean, zmode, and zmedian, and make comparisons between them in the next section, where we demonstrate that zbest indeed performs best. Equally important to the point estimate is the reliability of the point estimate, and we naturally use the risk parameter, R(zphot), for this.3 We compute the risk parameter for each point estimate [e.g., R(zmean)]. To facilitate comparisons with previous work, we also compute the commonly used estimator of redshift confidence, C(z), defined as   \begin{equation} C(z_{\rm phot})=\int _{z_{\rm phot}-0.03}^{z_{\rm phot}+0.03} P(z)dz, \end{equation} (15)where zphot is a point estimate such as median and best. This is primarily to maintain consistency with previous studies, since we will show later that R(zphot) is a better estimator of photo-z reliability. 5 Performance evaluation using point estimates We now characterize the performance of our photo-z’s. We first evaluate how well the “best” point estimator works compared to other popular statistics. We then move on to show our photo-z accuracy at the Wide depth, followed by discussions on the depth and seeing dependence of the accuracy. We focus on the point estimator to characterize our photo-z performance in this section. We present PDF-based tests in section 6. 5.1 The “best” point estimate One of the most popular point estimators used for photo-z is the median, which is defined as the redshift at which the integrated probability equals 0.5. The mode of the PDF is also frequently used. We compare the mean, mode, median, and best redshifts using the COSMOS Wide-depth median seeing stack (see the next section for details) for MLZ in table 1. We use all galaxies with i < 25 here. The best estimator gives the smallest scatter and lowest outlier rate4 compared to the other estimators. The best estimator tends to introduce a small negative bias, but the bias is not sufficiently large to prevent most scientific applications. The other photo-z codes show the same trend. Based on this result, we will use the best estimator in what follows and denote the best redshift as zphot for simplicity. Table 1. Photo-z performance using mean, mode, median, and best estimators.* Point estimator  Bias  σconv  foutlier, conv  σ  foutlier  L(Δz)  Mean  −0.003  0.075  0.227  0.078  0.218  0.260  Mode  −0.002  0.067  0.213  0.064  0.240  0.244  Median  −0.001  0.066  0.199  0.064  0.226  0.236  Best  −0.003  0.064  0.197  0.061  0.229  0.233  Point estimator  Bias  σconv  foutlier, conv  σ  foutlier  L(Δz)  Mean  −0.003  0.075  0.227  0.078  0.218  0.260  Mode  −0.002  0.067  0.213  0.064  0.240  0.244  Median  −0.001  0.066  0.199  0.064  0.226  0.236  Best  −0.003  0.064  0.197  0.061  0.229  0.233  *The numbers are for MLZ, but all the other codes show the same trend. View Large 5.2 Photo-z performance at the Wide depth We characterize the photo-z performance at the Wide depth, a representative depth of the HSC survey as a whole, using the metrics defined in section 4. We compare zphot with zref for the COSMOS Wide-depth median seeing stack. Recall that this is a subsample of UltraDeep COSMOS data stacked to the depth of the Wide survey, and all the filters have 0$${^{\prime\prime}_{.}}$$7 seeing. Figure 3 shows the bias, scatter, and outlier fraction as a function of i-band magnitude for all the codes. More statistics are summarized in table 2. Most of the reference redshifts at faint magnitudes come from the COSMOS photo-z catalog, and they are not very accurate at i ≳ 25. We thus cut at i = 25 and characterize the performance at brighter mags. Once again, not all the COSMOS photo-z’s are correct and we inherit the systematic uncertainty from COSMOS. The absolute numbers of the statistics shown in the figure should thus be taken with caution. Fig. 3. View largeDownload slide Bias, foutlier, conv, and σconv plotted against i-band magnitude. The different panels are for different codes, as indicated by the label on the top-left corner of each panel. The gray shades show a ±0.01 range, which will be useful for bias. The symbols are explained in the panels. Note that these plots are based on the COSMOS Wide-depth median stack and include objects in COSMOS only. (Color online) Fig. 3. View largeDownload slide Bias, foutlier, conv, and σconv plotted against i-band magnitude. The different panels are for different codes, as indicated by the label on the top-left corner of each panel. The gray shades show a ±0.01 range, which will be useful for bias. The symbols are explained in the panels. Note that these plots are based on the COSMOS Wide-depth median stack and include objects in COSMOS only. (Color online) Table 2. Photo-z statistics for all the codes as a function of magnitude.* Code  Magnitude  Bias  σconv  foutlier, conv  σ  foutlier  ⟨L(Δz)⟩  DEmP  18.50–18.75  −0.000  0.060  0.430  0.023  0.463  0.373    18.75–19.00  +0.003  0.026  0.124  0.025  0.210  0.124    19.00–19.25  −0.001  0.015  0.000  0.017  0.145  0.027    19.25–19.50  +0.002  0.024  0.050  0.021  0.156  0.070    19.50–19.75  +0.001  0.025  0.090  0.023  0.209  0.101    19.75–20.00  +0.004  0.022  0.108  0.020  0.245  0.112    20.00–20.25  +0.004  0.024  0.047  0.023  0.191  0.079    20.25–20.50  +0.004  0.021  0.039  0.019  0.178  0.061    20.50–20.75  +0.007  0.023  0.052  0.021  0.186  0.077    20.75–21.00  +0.005  0.021  0.037  0.020  0.166  0.058    21.00–21.25  +0.006  0.024  0.045  0.022  0.171  0.072    21.25–21.50  +0.005  0.022  0.037  0.021  0.182  0.066    21.50–21.75  +0.006  0.023  0.040  0.021  0.156  0.064    21.75–22.00  +0.005  0.024  0.044  0.024  0.165  0.073    22.00–22.25  +0.005  0.024  0.042  0.023  0.163  0.072    22.25–22.50  +0.004  0.025  0.049  0.024  0.166  0.078    22.50–22.75  +0.003  0.028  0.044  0.027  0.153  0.078    22.75–23.00  +0.001  0.030  0.045  0.030  0.150  0.082    23.00–23.25  +0.002  0.033  0.063  0.033  0.160  0.103    23.25–23.50  +0.000  0.039  0.089  0.038  0.173  0.128    23.50–23.75  −0.001  0.046  0.123  0.044  0.201  0.164    23.75–24.00  −0.001  0.057  0.167  0.054  0.217  0.208    24.00–24.25  −0.004  0.072  0.209  0.070  0.222  0.251    24.25–24.50  −0.007  0.090  0.261  0.089  0.227  0.297    24.50–24.75  −0.011  0.107  0.308  0.110  0.222  0.335    24.75–25.00  −0.014  0.121  0.331  0.127  0.208  0.357  EPHOR  18.50–18.75  −0.003  0.047  0.430  0.020  0.437  0.422    18.75–19.00  +0.007  0.028  0.124  0.017  0.271  0.139    19.00–19.25  −0.000  0.016  0.023  0.018  0.133  0.037    19.25–19.50  −0.004  0.023  0.040  0.024  0.201  0.077    19.50–19.75  −0.000  0.024  0.087  0.021  0.176  0.092    19.75–20.00  −0.002  0.026  0.073  0.024  0.191  0.099    20.00–20.25  +0.002  0.021  0.044  0.022  0.205  0.078    20.25–20.50  +0.001  0.021  0.052  0.020  0.164  0.071    20.50–20.75  +0.003  0.023  0.047  0.023  0.157  0.069    20.75–21.00  +0.002  0.022  0.040  0.021  0.157  0.065    21.00–21.25  +0.002  0.023  0.048  0.023  0.157  0.071    21.25–21.50  +0.001  0.023  0.037  0.024  0.144  0.064    21.50–21.75  +0.001  0.024  0.042  0.024  0.145  0.066    21.75–22.00  −0.000  0.025  0.053  0.026  0.165  0.078    22.00–22.25  −0.000  0.025  0.048  0.025  0.168  0.079    22.25–22.50  −0.001  0.026  0.056  0.026  0.172  0.084    22.50–22.75  −0.001  0.029  0.049  0.029  0.153  0.083    22.75–23.00  −0.002  0.031  0.056  0.031  0.159  0.092    23.00–23.25  −0.001  0.035  0.071  0.034  0.164  0.109    23.25–23.50  −0.003  0.040  0.101  0.037  0.183  0.137    23.50–23.75  −0.003  0.047  0.137  0.044  0.211  0.175    23.75–24.00  −0.004  0.057  0.184  0.052  0.235  0.217    24.00–24.25  −0.006  0.071  0.224  0.066  0.243  0.259    24.25–24.50  −0.011  0.087  0.259  0.085  0.235  0.293    24.50–24.75  −0.017  0.104  0.312  0.110  0.226  0.333    24.75–25.00  −0.024  0.119  0.335  0.125  0.210  0.356  EPHOR_AB  18.50–18.75  −0.000  0.005  0.000  0.008  0.325  0.014    18.75–19.00  −0.002  0.026  0.124  0.026  0.192  0.140    19.00–19.25  −0.005  0.017  0.000  0.019  0.123  0.035    19.25–19.50  −0.004  0.026  0.021  0.021  0.105  0.046    19.50–19.75  −0.004  0.028  0.098  0.023  0.169  0.100    19.75–20.00  −0.003  0.025  0.110  0.021  0.229  0.121    20.00–20.25  −0.001  0.024  0.053  0.026  0.174  0.082    20.25–20.50  −0.001  0.023  0.038  0.021  0.147  0.061    20.50–20.75  +0.002  0.025  0.053  0.022  0.163  0.076    20.75–21.00  +0.001  0.022  0.042  0.022  0.143  0.059    21.00–21.25  +0.001  0.023  0.059  0.022  0.178  0.080    21.25–21.50  +0.001  0.023  0.050  0.022  0.169  0.072    21.50–21.75  +0.001  0.023  0.047  0.022  0.153  0.066    21.75–22.00  +0.000  0.023  0.050  0.024  0.165  0.075    22.00–22.25  −0.000  0.025  0.052  0.024  0.157  0.078    22.25–22.50  −0.001  0.026  0.053  0.025  0.162  0.078    22.50–22.75  −0.002  0.028  0.051  0.027  0.147  0.080    22.75–23.00  −0.003  0.030  0.052  0.030  0.148  0.085    23.00–23.25  −0.003  0.033  0.069  0.032  0.155  0.102    23.25–23.50  −0.004  0.039  0.086  0.037  0.164  0.123    23.50–23.75  −0.005  0.047  0.113  0.044  0.182  0.155    23.75–24.00  −0.005  0.057  0.151  0.052  0.200  0.194    24.00–24.25  −0.005  0.069  0.194  0.068  0.209  0.237    24.25–24.50  −0.006  0.081  0.222  0.081  0.206  0.264    24.50–24.75  −0.007  0.095  0.263  0.097  0.215  0.303    24.75–25.00  −0.009  0.104  0.292  0.110  0.212  0.326  FRANKEN-Z  18.75–19.00  −0.000  0.021  0.000  0.028  0.120  0.039    19.00–19.25  −0.005  0.012  0.000  0.010  0.228  0.032    19.25–19.50  −0.001  0.013  0.011  0.015  0.189  0.036    19.50–19.75  −0.002  0.012  0.018  0.015  0.182  0.038    19.75–20.00  −0.001  0.015  0.015  0.017  0.145  0.043    20.00–20.25  −0.001  0.012  0.019  0.013  0.199  0.039    20.25–20.50  −0.002  0.013  0.024  0.014  0.200  0.043    20.50–20.75  −0.001  0.012  0.029  0.013  0.195  0.045    20.75–21.00  −0.002  0.013  0.022  0.014  0.162  0.038    21.00–21.25  −0.001  0.015  0.025  0.016  0.156  0.046    21.25–21.50  −0.001  0.014  0.033  0.015  0.178  0.046    21.50–21.75  −0.001  0.015  0.030  0.016  0.173  0.048    21.75–22.00  −0.001  0.019  0.042  0.019  0.173  0.060    22.00–22.25  −0.001  0.019  0.039  0.020  0.174  0.062    22.25–22.50  −0.001  0.022  0.047  0.022  0.173  0.069    22.50–22.75  −0.002  0.024  0.043  0.025  0.152  0.069    22.75–23.00  −0.003  0.027  0.046  0.027  0.158  0.079    23.00–23.25  −0.003  0.032  0.067  0.031  0.172  0.103    23.25–23.50  −0.003  0.035  0.088  0.034  0.180  0.122    23.50–23.75  −0.003  0.043  0.113  0.041  0.197  0.154    23.75–24.00  −0.003  0.054  0.153  0.050  0.214  0.195    24.00–24.25  −0.004  0.067  0.197  0.064  0.224  0.239    24.25–24.50  −0.007  0.080  0.234  0.079  0.223  0.273    24.50–24.75  −0.009  0.096  0.279  0.096  0.226  0.310    24.75–25.00  −0.009  0.106  0.303  0.112  0.217  0.332  Mizuki  18.50–18.75  −0.006  0.131  0.430  0.051  0.464  0.481    18.75–19.00  +0.000  0.032  0.124  0.036  0.278  0.190    19.00–19.25  −0.009  0.028  0.015  0.036  0.115  0.075    19.25–19.50  −0.004  0.035  0.066  0.036  0.161  0.112    19.50–19.75  +0.001  0.043  0.130  0.037  0.228  0.178    19.75–20.00  +0.006  0.041  0.132  0.041  0.237  0.185    20.00–20.25  +0.006  0.038  0.127  0.040  0.234  0.171    20.25–20.50  +0.004  0.033  0.086  0.030  0.210  0.122    20.50–20.75  +0.006  0.035  0.097  0.032  0.218  0.126    20.75–21.00  +0.003  0.032  0.078  0.033  0.179  0.110    21.00–21.25  +0.001  0.035  0.094  0.034  0.174  0.127    21.25–21.50  −0.001  0.032  0.107  0.029  0.208  0.134    21.50–21.75  +0.001  0.033  0.095  0.030  0.166  0.121    21.75–22.00  −0.001  0.036  0.100  0.033  0.184  0.132    22.00–22.25  −0.001  0.035  0.095  0.032  0.187  0.126    22.25–22.50  −0.001  0.036  0.099  0.034  0.184  0.131    22.50–22.75  −0.002  0.036  0.085  0.035  0.172  0.123    22.75–23.00  −0.002  0.041  0.093  0.039  0.171  0.136    23.00–23.25  +0.001  0.045  0.103  0.042  0.174  0.146    23.25–23.50  −0.000  0.051  0.126  0.048  0.186  0.170    23.50–23.75  −0.001  0.059  0.149  0.058  0.192  0.197    23.75–24.00  −0.002  0.071  0.198  0.070  0.212  0.244    24.00–24.25  −0.006  0.086  0.246  0.086  0.217  0.286    24.25–24.50  −0.008  0.103  0.275  0.102  0.212  0.316    24.50–24.75  −0.008  0.124  0.334  0.121  0.228  0.364    24.75–25.00  −0.011  0.141  0.362  0.137  0.222  0.391  MLZ  18.50–18.75  +0.014  0.087  0.430  0.424  0.000  0.435    18.75–19.00  −0.002  0.051  0.062  0.050  0.078  0.132    19.00–19.25  −0.002  0.038  0.008  0.046  0.062  0.076    19.25–19.50  −0.001  0.051  0.044  0.046  0.091  0.107    19.50–19.75  +0.006  0.054  0.090  0.049  0.154  0.153    19.75–20.00  +0.013  0.055  0.109  0.053  0.164  0.170    20.00–20.25  +0.012  0.052  0.065  0.056  0.127  0.142    20.25–20.50  +0.004  0.042  0.051  0.045  0.138  0.111    20.50–20.75  +0.008  0.045  0.091  0.041  0.159  0.125    20.75–21.00  +0.006  0.039  0.069  0.043  0.151  0.116    21.00–21.25  +0.006  0.039  0.068  0.041  0.146  0.116    21.25–21.50  +0.005  0.038  0.074  0.037  0.170  0.111    21.50–21.75  +0.006  0.039  0.069  0.037  0.135  0.106    21.75–22.00  +0.004  0.039  0.073  0.037  0.155  0.112    22.00–22.25  +0.002  0.038  0.075  0.038  0.163  0.115    22.25–22.50  +0.000  0.039  0.066  0.039  0.147  0.109    22.50–22.75  −0.001  0.038  0.067  0.039  0.143  0.110    22.75–23.00  −0.004  0.040  0.067  0.040  0.148  0.112    23.00–23.25  −0.002  0.043  0.088  0.042  0.163  0.130    23.25–23.50  −0.003  0.046  0.106  0.044  0.175  0.148    23.50–23.75  −0.004  0.054  0.138  0.052  0.194  0.183    23.75–24.00  −0.005  0.065  0.184  0.062  0.214  0.225    24.00–24.25  −0.005  0.077  0.230  0.075  0.229  0.265    24.25–24.50  −0.007  0.092  0.274  0.095  0.229  0.306    24.50–24.75  −0.009  0.107  0.317  0.110  0.234  0.341    24.75–25.00  −0.012  0.119  0.343  0.126  0.228  0.364  NNPZ  18.50–18.75  +0.004  0.007  0.000  0.009  0.379  0.020    18.75–19.00  −0.001  0.034  0.000  0.031  0.079  0.055    19.00–19.25  −0.001  0.027  0.008  0.032  0.094  0.059    19.25–19.50  −0.005  0.039  0.010  0.039  0.079  0.070    19.50–19.75  +0.001  0.028  0.040  0.029  0.127  0.073    19.75–20.00  +0.002  0.031  0.048  0.031  0.150  0.084    20.00–20.25  +0.004  0.032  0.046  0.034  0.144  0.094    20.25–20.50  +0.000  0.027  0.042  0.027  0.164  0.075    20.50–20.75  +0.001  0.031  0.059  0.030  0.166  0.089    20.75–21.00  −0.000  0.027  0.048  0.027  0.161  0.078    21.00–21.25  −0.000  0.027  0.051  0.027  0.162  0.081    21.25–21.50  −0.000  0.027  0.046  0.028  0.148  0.072    21.50–21.75  −0.000  0.027  0.039  0.026  0.149  0.071    21.75–22.00  −0.001  0.028  0.051  0.028  0.153  0.083    22.00–22.25  −0.001  0.029  0.056  0.028  0.163  0.088    22.25–22.50  −0.001  0.030  0.061  0.030  0.161  0.092    22.50–22.75  −0.001  0.032  0.056  0.031  0.148  0.091    22.75–23.00  −0.002  0.035  0.061  0.035  0.151  0.101    23.00–23.25  −0.002  0.038  0.076  0.038  0.164  0.119    23.25–23.50  −0.003  0.042  0.106  0.040  0.187  0.145    23.50–23.75  −0.003  0.050  0.138  0.048  0.208  0.178    23.75–24.00  −0.004  0.061  0.185  0.057  0.227  0.220    24.00–24.25  −0.007  0.074  0.231  0.071  0.239  0.264    24.25–24.50  −0.011  0.089  0.275  0.090  0.239  0.303    24.50–24.75  −0.015  0.104  0.319  0.112  0.225  0.337    24.75–25.00  −0.017  0.115  0.333  0.122  0.219  0.354  Code  Magnitude  Bias  σconv  foutlier, conv  σ  foutlier  ⟨L(Δz)⟩  DEmP  18.50–18.75  −0.000  0.060  0.430  0.023  0.463  0.373    18.75–19.00  +0.003  0.026  0.124  0.025  0.210  0.124    19.00–19.25  −0.001  0.015  0.000  0.017  0.145  0.027    19.25–19.50  +0.002  0.024  0.050  0.021  0.156  0.070    19.50–19.75  +0.001  0.025  0.090  0.023  0.209  0.101    19.75–20.00  +0.004  0.022  0.108  0.020  0.245  0.112    20.00–20.25  +0.004  0.024  0.047  0.023  0.191  0.079    20.25–20.50  +0.004  0.021  0.039  0.019  0.178  0.061    20.50–20.75  +0.007  0.023  0.052  0.021  0.186  0.077    20.75–21.00  +0.005  0.021  0.037  0.020  0.166  0.058    21.00–21.25  +0.006  0.024  0.045  0.022  0.171  0.072    21.25–21.50  +0.005  0.022  0.037  0.021  0.182  0.066    21.50–21.75  +0.006  0.023  0.040  0.021  0.156  0.064    21.75–22.00  +0.005  0.024  0.044  0.024  0.165  0.073    22.00–22.25  +0.005  0.024  0.042  0.023  0.163  0.072    22.25–22.50  +0.004  0.025  0.049  0.024  0.166  0.078    22.50–22.75  +0.003  0.028  0.044  0.027  0.153  0.078    22.75–23.00  +0.001  0.030  0.045  0.030  0.150  0.082    23.00–23.25  +0.002  0.033  0.063  0.033  0.160  0.103    23.25–23.50  +0.000  0.039  0.089  0.038  0.173  0.128    23.50–23.75  −0.001  0.046  0.123  0.044  0.201  0.164    23.75–24.00  −0.001  0.057  0.167  0.054  0.217  0.208    24.00–24.25  −0.004  0.072  0.209  0.070  0.222  0.251    24.25–24.50  −0.007  0.090  0.261  0.089  0.227  0.297    24.50–24.75  −0.011  0.107  0.308  0.110  0.222  0.335    24.75–25.00  −0.014  0.121  0.331  0.127  0.208  0.357  EPHOR  18.50–18.75  −0.003  0.047  0.430  0.020  0.437  0.422    18.75–19.00  +0.007  0.028  0.124  0.017  0.271  0.139    19.00–19.25  −0.000  0.016  0.023  0.018  0.133  0.037    19.25–19.50  −0.004  0.023  0.040  0.024  0.201  0.077    19.50–19.75  −0.000  0.024  0.087  0.021  0.176  0.092    19.75–20.00  −0.002  0.026  0.073  0.024  0.191  0.099    20.00–20.25  +0.002  0.021  0.044  0.022  0.205  0.078    20.25–20.50  +0.001  0.021  0.052  0.020  0.164  0.071    20.50–20.75  +0.003  0.023  0.047  0.023  0.157  0.069    20.75–21.00  +0.002  0.022  0.040  0.021  0.157  0.065    21.00–21.25  +0.002  0.023  0.048  0.023  0.157  0.071    21.25–21.50  +0.001  0.023  0.037  0.024  0.144  0.064    21.50–21.75  +0.001  0.024  0.042  0.024  0.145  0.066    21.75–22.00  −0.000  0.025  0.053  0.026  0.165  0.078    22.00–22.25  −0.000  0.025  0.048  0.025  0.168  0.079    22.25–22.50  −0.001  0.026  0.056  0.026  0.172  0.084    22.50–22.75  −0.001  0.029  0.049  0.029  0.153  0.083    22.75–23.00  −0.002  0.031  0.056  0.031  0.159  0.092    23.00–23.25  −0.001  0.035  0.071  0.034  0.164  0.109    23.25–23.50  −0.003  0.040  0.101  0.037  0.183  0.137    23.50–23.75  −0.003  0.047  0.137  0.044  0.211  0.175    23.75–24.00  −0.004  0.057  0.184  0.052  0.235  0.217    24.00–24.25  −0.006  0.071  0.224  0.066  0.243  0.259    24.25–24.50  −0.011  0.087  0.259  0.085  0.235  0.293    24.50–24.75  −0.017  0.104  0.312  0.110  0.226  0.333    24.75–25.00  −0.024  0.119  0.335  0.125  0.210  0.356  EPHOR_AB  18.50–18.75  −0.000  0.005  0.000  0.008  0.325  0.014    18.75–19.00  −0.002  0.026  0.124  0.026  0.192  0.140    19.00–19.25  −0.005  0.017  0.000  0.019  0.123  0.035    19.25–19.50  −0.004  0.026  0.021  0.021  0.105  0.046    19.50–19.75  −0.004  0.028  0.098  0.023  0.169  0.100    19.75–20.00  −0.003  0.025  0.110  0.021  0.229  0.121    20.00–20.25  −0.001  0.024  0.053  0.026  0.174  0.082    20.25–20.50  −0.001  0.023  0.038  0.021  0.147  0.061    20.50–20.75  +0.002  0.025  0.053  0.022  0.163  0.076    20.75–21.00  +0.001  0.022  0.042  0.022  0.143  0.059    21.00–21.25  +0.001  0.023  0.059  0.022  0.178  0.080    21.25–21.50  +0.001  0.023  0.050  0.022  0.169  0.072    21.50–21.75  +0.001  0.023  0.047  0.022  0.153  0.066    21.75–22.00  +0.000  0.023  0.050  0.024  0.165  0.075    22.00–22.25  −0.000  0.025  0.052  0.024  0.157  0.078    22.25–22.50  −0.001  0.026  0.053  0.025  0.162  0.078    22.50–22.75  −0.002  0.028  0.051  0.027  0.147  0.080    22.75–23.00  −0.003  0.030  0.052  0.030  0.148  0.085    23.00–23.25  −0.003  0.033  0.069  0.032  0.155  0.102    23.25–23.50  −0.004  0.039  0.086  0.037  0.164  0.123    23.50–23.75  −0.005  0.047  0.113  0.044  0.182  0.155    23.75–24.00  −0.005  0.057  0.151  0.052  0.200  0.194    24.00–24.25  −0.005  0.069  0.194  0.068  0.209  0.237    24.25–24.50  −0.006  0.081  0.222  0.081  0.206  0.264    24.50–24.75  −0.007  0.095  0.263  0.097  0.215  0.303    24.75–25.00  −0.009  0.104  0.292  0.110  0.212  0.326  FRANKEN-Z  18.75–19.00  −0.000  0.021  0.000  0.028  0.120  0.039    19.00–19.25  −0.005  0.012  0.000  0.010  0.228  0.032    19.25–19.50  −0.001  0.013  0.011  0.015  0.189  0.036    19.50–19.75  −0.002  0.012  0.018  0.015  0.182  0.038    19.75–20.00  −0.001  0.015  0.015  0.017  0.145  0.043    20.00–20.25  −0.001  0.012  0.019  0.013  0.199  0.039    20.25–20.50  −0.002  0.013  0.024  0.014  0.200  0.043    20.50–20.75  −0.001  0.012  0.029  0.013  0.195  0.045    20.75–21.00  −0.002  0.013  0.022  0.014  0.162  0.038    21.00–21.25  −0.001  0.015  0.025  0.016  0.156  0.046    21.25–21.50  −0.001  0.014  0.033  0.015  0.178  0.046    21.50–21.75  −0.001  0.015  0.030  0.016  0.173  0.048    21.75–22.00  −0.001  0.019  0.042  0.019  0.173  0.060    22.00–22.25  −0.001  0.019  0.039  0.020  0.174  0.062    22.25–22.50  −0.001  0.022  0.047  0.022  0.173  0.069    22.50–22.75  −0.002  0.024  0.043  0.025  0.152  0.069    22.75–23.00  −0.003  0.027  0.046  0.027  0.158  0.079    23.00–23.25  −0.003  0.032  0.067  0.031  0.172  0.103    23.25–23.50  −0.003  0.035  0.088  0.034  0.180  0.122    23.50–23.75  −0.003  0.043  0.113  0.041  0.197  0.154    23.75–24.00  −0.003  0.054  0.153  0.050  0.214  0.195    24.00–24.25  −0.004  0.067  0.197  0.064  0.224  0.239    24.25–24.50  −0.007  0.080  0.234  0.079  0.223  0.273    24.50–24.75  −0.009  0.096  0.279  0.096  0.226  0.310    24.75–25.00  −0.009  0.106  0.303  0.112  0.217  0.332  Mizuki  18.50–18.75  −0.006  0.131  0.430  0.051  0.464  0.481    18.75–19.00  +0.000  0.032  0.124  0.036  0.278  0.190    19.00–19.25  −0.009  0.028  0.015  0.036  0.115  0.075    19.25–19.50  −0.004  0.035  0.066  0.036  0.161  0.112    19.50–19.75  +0.001  0.043  0.130  0.037  0.228  0.178    19.75–20.00  +0.006  0.041  0.132  0.041  0.237  0.185    20.00–20.25  +0.006  0.038  0.127  0.040  0.234  0.171    20.25–20.50  +0.004  0.033  0.086  0.030  0.210  0.122    20.50–20.75  +0.006  0.035  0.097  0.032  0.218  0.126    20.75–21.00  +0.003  0.032  0.078  0.033  0.179  0.110    21.00–21.25  +0.001  0.035  0.094  0.034  0.174  0.127    21.25–21.50  −0.001  0.032  0.107  0.029  0.208  0.134    21.50–21.75  +0.001  0.033  0.095  0.030  0.166  0.121    21.75–22.00  −0.001  0.036  0.100  0.033  0.184  0.132    22.00–22.25  −0.001  0.035  0.095  0.032  0.187  0.126    22.25–22.50  −0.001  0.036  0.099  0.034  0.184  0.131    22.50–22.75  −0.002  0.036  0.085  0.035  0.172  0.123    22.75–23.00  −0.002  0.041  0.093  0.039  0.171  0.136    23.00–23.25  +0.001  0.045  0.103  0.042  0.174  0.146    23.25–23.50  −0.000  0.051  0.126  0.048  0.186  0.170    23.50–23.75  −0.001  0.059  0.149  0.058  0.192  0.197    23.75–24.00  −0.002  0.071  0.198  0.070  0.212  0.244    24.00–24.25  −0.006  0.086  0.246  0.086  0.217  0.286    24.25–24.50  −0.008  0.103  0.275  0.102  0.212  0.316    24.50–24.75  −0.008  0.124  0.334  0.121  0.228  0.364    24.75–25.00  −0.011  0.141  0.362  0.137  0.222  0.391  MLZ  18.50–18.75  +0.014  0.087  0.430  0.424  0.000  0.435    18.75–19.00  −0.002  0.051  0.062  0.050  0.078  0.132    19.00–19.25  −0.002  0.038  0.008  0.046  0.062  0.076    19.25–19.50  −0.001  0.051  0.044  0.046  0.091  0.107    19.50–19.75  +0.006  0.054  0.090  0.049  0.154  0.153    19.75–20.00  +0.013  0.055  0.109  0.053  0.164  0.170    20.00–20.25  +0.012  0.052  0.065  0.056  0.127  0.142    20.25–20.50  +0.004  0.042  0.051  0.045  0.138  0.111    20.50–20.75  +0.008  0.045  0.091  0.041  0.159  0.125    20.75–21.00  +0.006  0.039  0.069  0.043  0.151  0.116    21.00–21.25  +0.006  0.039  0.068  0.041  0.146  0.116    21.25–21.50  +0.005  0.038  0.074  0.037  0.170  0.111    21.50–21.75  +0.006  0.039  0.069  0.037  0.135  0.106    21.75–22.00  +0.004  0.039  0.073  0.037  0.155  0.112    22.00–22.25  +0.002  0.038  0.075  0.038  0.163  0.115    22.25–22.50  +0.000  0.039  0.066  0.039  0.147  0.109    22.50–22.75  −0.001  0.038  0.067  0.039  0.143  0.110    22.75–23.00  −0.004  0.040  0.067  0.040  0.148  0.112    23.00–23.25  −0.002  0.043  0.088  0.042  0.163  0.130    23.25–23.50  −0.003  0.046  0.106  0.044  0.175  0.148    23.50–23.75  −0.004  0.054  0.138  0.052  0.194  0.183    23.75–24.00  −0.005  0.065  0.184  0.062  0.214  0.225    24.00–24.25  −0.005  0.077  0.230  0.075  0.229  0.265    24.25–24.50  −0.007  0.092  0.274  0.095  0.229  0.306    24.50–24.75  −0.009  0.107  0.317  0.110  0.234  0.341    24.75–25.00  −0.012  0.119  0.343  0.126  0.228  0.364  NNPZ  18.50–18.75  +0.004  0.007  0.000  0.009  0.379  0.020    18.75–19.00  −0.001  0.034  0.000  0.031  0.079  0.055    19.00–19.25  −0.001  0.027  0.008  0.032  0.094  0.059    19.25–19.50  −0.005  0.039  0.010  0.039  0.079  0.070    19.50–19.75  +0.001  0.028  0.040  0.029  0.127  0.073    19.75–20.00  +0.002  0.031  0.048  0.031  0.150  0.084    20.00–20.25  +0.004  0.032  0.046  0.034  0.144  0.094    20.25–20.50  +0.000  0.027  0.042  0.027  0.164  0.075    20.50–20.75  +0.001  0.031  0.059  0.030  0.166  0.089    20.75–21.00  −0.000  0.027  0.048  0.027  0.161  0.078    21.00–21.25  −0.000  0.027  0.051  0.027  0.162  0.081    21.25–21.50  −0.000  0.027  0.046  0.028  0.148  0.072    21.50–21.75  −0.000  0.027  0.039  0.026  0.149  0.071    21.75–22.00  −0.001  0.028  0.051  0.028  0.153  0.083    22.00–22.25  −0.001  0.029  0.056  0.028  0.163  0.088    22.25–22.50  −0.001  0.030  0.061  0.030  0.161  0.092    22.50–22.75  −0.001  0.032  0.056  0.031  0.148  0.091    22.75–23.00  −0.002  0.035  0.061  0.035  0.151  0.101    23.00–23.25  −0.002  0.038  0.076  0.038  0.164  0.119    23.25–23.50  −0.003  0.042  0.106  0.040  0.187  0.145    23.50–23.75  −0.003  0.050  0.138  0.048  0.208  0.178    23.75–24.00  −0.004  0.061  0.185  0.057  0.227  0.220    24.00–24.25  −0.007  0.074  0.231  0.071  0.239  0.264    24.25–24.50  −0.011  0.089  0.275  0.090  0.239  0.303    24.50–24.75  −0.015  0.104  0.319  0.112  0.225  0.337    24.75–25.00  −0.017  0.115  0.333  0.122  0.219  0.354  *The numbers are for all galaxies down to i = 25. View Large Before we compare the codes, it is important to note that we observe a sign of of over-fitting in FRANKEN-Z even though the COSMOS Wide-depth stacks were not explicitly included during the training. This is likely due to FRANKEN-Z’s sensitivity to both the redshift PDF and error distributions in both datasets, which makes it more sensitive to our assumption that the two datasets are quasi-independent (see section 6 for details). It is thus unfair to compare its performance directly with the other codes, since it is likely to be overly optimistic. The photo-z accuracy is a strong function of magnitude, but it is relatively flat down to i ∼ 23 for all the codes. The scatter and outlier rate are about 0.03 and 5%, respectively, at such bright magnitudes. This flat performance is likely because most objects within this magnitude range are located at z ≲ 1.5, where we can obtain fairly good photo-z’s with the grizy photometry (see below). At fainter mags, the fraction of z > 1.5 objects increases and these high-redshift galaxies drive the poor performance at faint mags. It is encouraging that the bias is still within ∼1% at all mags for most codes. Figure 4 shows the same metrics but as a function of zphot. Our performance is poor at both the low-z (z ≲ 0.2) and high-z (z ≳ 1.5) ends. This is expected, because our filter set (grizy) does not straddle the 4000 Å break at these redshifts. We are not able to break the degeneracy between z ∼ 0 and z ∼ 2 solutions (i.e., 4000 Å break and Lyman break degeneracy), resulting in poor performance at low-z. At z ≳ 1.5, we probe only the featureless UV continuum and it makes it difficult to obtain good photo-z’s there. The Lyman break comes in the g band and some codes show improvements at z ≳ 3.5. In the good redshift range (0.2 ≲ z ≲ 1.5), our photo-z’s are fair—the outlier rate is about 15% and the scatter is about 0.05. Note that these numbers are for all galaxies down to i = 25. If we use brighter galaxies with i < 24, the numbers improve to about 8% and 0.04. Our photo-z’s should thus be sufficient to enable many of the science goals in HSC-SSP. Also, we can clip potential photo-z outliers to further improve the accuracy (see subsection 5.5). Table 3 summarizes the statistical measures for all the codes. Fig. 4. View largeDownload slide As figure 3, but as a function of zphot. (Color online) Fig. 4. View largeDownload slide As figure 3, but as a function of zphot. (Color online) Table 3. Photo-z statistics for all the codes as a function of zphot. Code  zphot  Bias  σconv  foutlier, conv  σ  foutlier  ⟨L(Δz)⟩  DEmP  0.00–0.10  +0.014  0.049  0.213  0.031  0.260  0.221    0.10–0.20  +0.004  0.044  0.112  0.042  0.168  0.157    0.20–0.30  +0.004  0.049  0.192  0.046  0.269  0.219    0.30–0.40  +0.002  0.042  0.135  0.042  0.238  0.181    0.40–0.50  +0.003  0.057  0.170  0.053  0.220  0.213    0.50–0.60  +0.007  0.055  0.200  0.047  0.256  0.231    0.60–0.70  +0.004  0.037  0.132  0.033  0.228  0.163    0.70–0.80  +0.003  0.041  0.115  0.042  0.210  0.154    0.80–0.90  −0.006  0.047  0.100  0.049  0.171  0.149    0.90–1.00  −0.000  0.040  0.148  0.038  0.239  0.165    1.00–1.10  −0.009  0.067  0.213  0.090  0.162  0.225    1.10–1.20  −0.006  0.050  0.141  0.052  0.203  0.171    1.20–1.30  −0.001  0.054  0.115  0.055  0.164  0.160    1.30–1.40  −0.001  0.056  0.123  0.061  0.163  0.172    1.40–1.50  −0.006  0.057  0.202  0.062  0.239  0.218    1.50–1.60  −0.007  0.070  0.268  0.091  0.231  0.268    1.60–1.70  −0.010  0.106  0.301  0.142  0.115  0.316    1.70–1.80  +0.000  0.113  0.273  0.132  0.136  0.319    1.80–1.90  +0.011  0.138  0.291  0.140  0.119  0.345    1.90–2.00  +0.003  0.124  0.280  0.118  0.117  0.323    2.00–2.10  +0.017  0.143  0.308  0.130  0.134  0.357    2.10–2.20  +0.014  0.154  0.330  0.139  0.119  0.373    2.20–2.30  +0.019  0.149  0.342  0.141  0.141  0.380    2.30–2.40  +0.020  0.153  0.337  0.140  0.166  0.387    2.40–2.50  +0.021  0.149  0.370  0.150  0.186  0.402    2.50–2.60  +0.014  0.132  0.342  0.122  0.250  0.388    2.60–2.70  −0.002  0.130  0.356  0.101  0.314  0.407    2.70–2.80  +0.007  0.120  0.334  0.086  0.307  0.382    2.80–2.90  +0.001  0.108  0.327  0.073  0.328  0.386    2.90–3.00  −0.001  0.109  0.364  0.067  0.373  0.406    3.00–3.10  +0.004  0.114  0.374  0.065  0.382  0.430    3.10–3.20  −0.005  0.129  0.434  0.059  0.460  0.482    3.20–3.30  −0.002  0.135  0.442  0.059  0.464  0.483    3.30–3.40  −0.008  0.117  0.406  0.051  0.435  0.450    3.40–3.50  −0.006  0.094  0.337  0.053  0.372  0.391    3.50–3.60  +0.002  0.070  0.312  0.044  0.361  0.353    3.60–3.70  −0.002  0.073  0.305  0.046  0.331  0.356    3.70–3.80  +0.007  0.086  0.321  0.052  0.337  0.374    3.80–3.90  +0.016  0.101  0.344  0.057  0.357  0.401    3.90–4.00  +0.024  0.099  0.313  0.058  0.322  0.377  EPHOR  0.00–0.10  −0.045  0.092  0.408  0.338  0.003  0.420    0.10–0.20  +0.002  0.054  0.155  0.048  0.202  0.207    0.20–0.30  +0.002  0.052  0.190  0.049  0.268  0.219    0.30–0.40  −0.003  0.042  0.152  0.039  0.253  0.194    0.40–0.50  −0.000  0.052  0.181  0.046  0.248  0.216    0.50–0.60  +0.005  0.065  0.251  0.050  0.296  0.274    0.60–0.70  +0.000  0.040  0.161  0.034  0.254  0.188    0.70–0.80  −0.004  0.046  0.157  0.044  0.237  0.188    0.80–0.90  −0.012  0.047  0.101  0.046  0.176  0.149    0.90–1.00  −0.003  0.045  0.182  0.040  0.262  0.190    1.00–1.10  −0.017  0.064  0.215  0.090  0.159  0.223    1.10–1.20  −0.016  0.053  0.147  0.052  0.192  0.178    1.20–1.30  −0.006  0.049  0.083  0.048  0.139  0.132    1.30–1.40  −0.002  0.049  0.094  0.051  0.155  0.143    1.40–1.50  −0.004  0.052  0.172  0.051  0.237  0.193    1.50–1.60  −0.000  0.065  0.262  0.067  0.280  0.262    1.60–1.70  −0.007  0.109  0.316  0.152  0.111  0.327    1.70–1.80  −0.009  0.127  0.313  0.147  0.114  0.341    1.80–1.90  +0.002  0.119  0.249  0.122  0.112  0.304    1.90–2.00  +0.002  0.133  0.274  0.124  0.098  0.322    2.00–2.10  +0.035  0.142  0.294  0.133  0.100  0.344    2.10–2.20  +0.032  0.137  0.343  0.140  0.110  0.354    2.20–2.30  +0.034  0.155  0.338  0.148  0.120  0.385    2.30–2.40  +0.007  0.134  0.312  0.128  0.204  0.368    2.40–2.50  +0.021  0.142  0.349  0.138  0.201  0.388    2.50–2.60  +0.033  0.132  0.370  0.143  0.212  0.389    2.60–2.70  +0.039  0.121  0.338  0.127  0.221  0.377    2.70–2.80  +0.020  0.118  0.275  0.092  0.221  0.344    2.80–2.90  +0.010  0.095  0.281  0.076  0.270  0.343    2.90–3.00  +0.007  0.090  0.312  0.069  0.315  0.353    3.00–3.10  +0.011  0.126  0.368  0.080  0.354  0.423    3.10–3.20  +0.004  0.113  0.364  0.070  0.370  0.419    3.20–3.30  +0.005  0.125  0.406  0.069  0.409  0.453    3.30–3.40  −0.001  0.104  0.354  0.062  0.379  0.412    3.40–3.50  +0.004  0.099  0.364  0.054  0.405  0.410    3.50–3.60  +0.007  0.062  0.262  0.042  0.316  0.304    3.60–3.70  +0.004  0.072  0.204  0.053  0.230  0.276    3.70–3.80  +0.000  0.081  0.281  0.057  0.299  0.340    3.80–3.90  +0.016  0.091  0.274  0.058  0.289  0.342    3.90–4.00  −0.016  0.087  0.380  0.039  0.441  0.419  EPHOR_AB  0.00–0.10  +0.001  0.068  0.218  0.054  0.248  0.254    0.10–0.20  −0.005  0.044  0.125  0.040  0.185  0.168    0.20–0.30  −0.004  0.051  0.190  0.048  0.266  0.221    0.30–0.40  −0.002  0.034  0.106  0.032  0.206  0.140    0.40–0.50  +0.004  0.058  0.187  0.052  0.232  0.226    0.50–0.60  +0.001  0.057  0.198  0.048  0.254  0.229    0.60–0.70  −0.003  0.036  0.105  0.033  0.202  0.139    0.70–0.80  +0.001  0.038  0.076  0.041  0.166  0.124    0.80–0.90  −0.005  0.045  0.080  0.044  0.152  0.129    0.90–1.00  −0.003  0.043  0.144  0.041  0.225  0.166    1.00–1.10  −0.019  0.075  0.237  0.106  0.108  0.242    1.10–1.20  −0.006  0.051  0.135  0.053  0.194  0.167    1.20–1.30  −0.004  0.055  0.101  0.053  0.146  0.149    1.30–1.40  −0.007  0.056  0.115  0.058  0.159  0.164    1.40–1.50  −0.012  0.057  0.190  0.054  0.255  0.209    1.50–1.60  −0.005  0.064  0.225  0.078  0.214  0.236    1.60–1.70  −0.010  0.098  0.266  0.128  0.132  0.288    1.70–1.80  −0.013  0.111  0.250  0.126  0.104  0.297    1.80–1.90  −0.011  0.118  0.232  0.118  0.091  0.293    1.90–2.00  −0.009  0.118  0.250  0.112  0.092  0.296    2.00–2.10  −0.011  0.118  0.211  0.110  0.068  0.281    2.10–2.20  −0.005  0.114  0.208  0.106  0.095  0.282    2.20–2.30  +0.002  0.129  0.256  0.124  0.119  0.319    2.30–2.40  +0.008  0.124  0.270  0.134  0.110  0.329    2.40–2.50  +0.014  0.124  0.290  0.134  0.176  0.348    2.50–2.60  −0.013  0.113  0.279  0.099  0.252  0.344    2.60–2.70  −0.003  0.114  0.308  0.088  0.287  0.370    2.70–2.80  −0.006  0.111  0.329  0.072  0.336  0.384    2.80–2.90  −0.009  0.111  0.363  0.067  0.390  0.414    2.90–3.00  −0.010  0.109  0.369  0.064  0.383  0.416    3.00–3.10  −0.007  0.103  0.359  0.058  0.384  0.418    3.10–3.20  −0.001  0.109  0.405  0.057  0.431  0.448    3.20–3.30  −0.008  0.100  0.351  0.058  0.385  0.406    3.30–3.40  −0.005  0.078  0.312  0.050  0.360  0.364    3.40–3.50  −0.003  0.066  0.274  0.046  0.308  0.322    3.50–3.60  −0.005  0.057  0.230  0.043  0.280  0.278    3.60–3.70  −0.010  0.074  0.232  0.050  0.255  0.293    3.70–3.80  −0.003  0.080  0.291  0.053  0.318  0.339    3.80–3.90  −0.005  0.095  0.303  0.058  0.320  0.368    3.90–4.00  −0.008  0.080  0.209  0.054  0.215  0.283  FRANKEN-Z  0.00–0.10  +0.011  0.044  0.281  0.021  0.402  0.282    0.10–0.20  −0.006  0.045  0.144  0.039  0.207  0.182    0.20–0.30  −0.003  0.044  0.198  0.036  0.303  0.225    0.30–0.40  −0.002  0.035  0.119  0.033  0.233  0.157    0.40–0.50  −0.001  0.053  0.162  0.048  0.230  0.204    0.50–0.60  +0.002  0.052  0.193  0.045  0.261  0.223    0.60–0.70  −0.001  0.036  0.125  0.033  0.225  0.156    0.70–0.80  −0.002  0.039  0.101  0.040  0.196  0.140    0.80–0.90  −0.008  0.044  0.086  0.044  0.169  0.135    0.90–1.00  −0.002  0.040  0.147  0.037  0.236  0.162    1.00–1.10  −0.008  0.060  0.198  0.074  0.195  0.211    1.10–1.20  −0.007  0.045  0.119  0.045  0.198  0.153    1.20–1.30  −0.003  0.049  0.091  0.049  0.151  0.139    1.30–1.40  −0.004  0.050  0.109  0.052  0.175  0.153    1.40–1.50  −0.006  0.051  0.191  0.050  0.262  0.206    1.50–1.60  −0.005  0.062  0.245  0.062  0.275  0.251    1.60–1.70  −0.012  0.095  0.268  0.122  0.145  0.288    1.70–1.80  +0.008  0.105  0.237  0.120  0.139  0.295    1.80–1.90  +0.000  0.116  0.249  0.122  0.116  0.302    1.90–2.00  −0.005  0.117  0.257  0.117  0.101  0.301    2.00–2.10  +0.002  0.130  0.276  0.120  0.115  0.321    2.10–2.20  +0.014  0.129  0.278  0.121  0.115  0.329    2.20–2.30  +0.018  0.147  0.291  0.138  0.119  0.353    2.30–2.40  +0.024  0.147  0.326  0.146  0.144  0.378    2.40–2.50  +0.019  0.121  0.334  0.132  0.203  0.356    2.50–2.60  +0.016  0.128  0.332  0.132  0.239  0.380    2.60–2.70  +0.005  0.111  0.297  0.093  0.270  0.354    2.70–2.80  −0.001  0.098  0.310  0.075  0.310  0.356    2.80–2.90  +0.001  0.105  0.328  0.070  0.338  0.377    2.90–3.00  +0.007  0.105  0.324  0.072  0.324  0.378    3.00–3.10  +0.004  0.096  0.340  0.059  0.354  0.393    3.10–3.20  +0.007  0.098  0.364  0.055  0.384  0.413    3.20–3.30  +0.006  0.116  0.421  0.053  0.444  0.465    3.30–3.40  +0.001  0.092  0.387  0.047  0.430  0.421    3.40–3.50  −0.001  0.078  0.299  0.049  0.348  0.350    3.50–3.60  +0.004  0.064  0.277  0.042  0.324  0.319    3.60–3.70  +0.007  0.065  0.237  0.046  0.273  0.293    3.70–3.80  +0.003  0.079  0.272  0.052  0.295  0.323    3.80–3.90  +0.013  0.092  0.334  0.050  0.344  0.376    3.90–4.00  +0.010  0.084  0.294  0.054  0.313  0.348  Mizuki  0.00–0.10  −0.272  0.286  0.550  0.330  0.000  0.522    0.10–0.20  +0.001  0.070  0.283  0.048  0.317  0.301    0.20–0.30  +0.007  0.059  0.193  0.055  0.257  0.231    0.30–0.40  +0.005  0.042  0.132  0.039  0.224  0.173    0.40–0.50  +0.013  0.074  0.244  0.061  0.270  0.287    0.50–0.60  +0.016  0.081  0.258  0.073  0.246  0.290    0.60–0.70  −0.001  0.047  0.137  0.044  0.218  0.177    0.70–0.80  −0.010  0.053  0.088  0.054  0.138  0.152    0.80–0.90  −0.012  0.053  0.095  0.050  0.161  0.151    0.90–1.00  −0.002  0.055  0.144  0.055  0.192  0.184    1.00–1.10  −0.028  0.088  0.239  0.111  0.091  0.259    1.10–1.20  −0.019  0.064  0.146  0.076  0.134  0.195    1.20–1.30  −0.003  0.061  0.101  0.062  0.137  0.162    1.30–1.40  −0.003  0.064  0.156  0.067  0.179  0.204    1.40–1.50  −0.006  0.064  0.228  0.069  0.244  0.242    1.50–1.60  +0.000  0.074  0.304  0.107  0.243  0.298    1.60–1.70  +0.012  0.113  0.329  0.153  0.101  0.332    1.70–1.80  +0.043  0.161  0.369  0.175  0.077  0.394    1.80–1.90  +0.033  0.169  0.385  0.172  0.080  0.398    1.90–2.00  +0.017  0.160  0.361  0.160  0.117  0.383    2.00–2.10  +0.019  0.161  0.341  0.149  0.102  0.376    2.10–2.20  +0.009  0.148  0.297  0.139  0.108  0.361    2.20–2.30  +0.024  0.166  0.347  0.153  0.128  0.404    2.30–2.40  +0.028  0.166  0.369  0.161  0.157  0.420    2.40–2.50  +0.011  0.146  0.362  0.148  0.199  0.405    2.50–2.60  +0.029  0.143  0.389  0.150  0.213  0.413    2.60–2.70  +0.019  0.142  0.385  0.132  0.285  0.426    2.70–2.80  +0.000  0.141  0.390  0.101  0.355  0.433    2.80–2.90  −0.004  0.165  0.445  0.081  0.439  0.491    2.90–3.00  +0.006  0.181  0.483  0.076  0.479  0.508    3.00–3.10  +0.006  0.130  0.435  0.062  0.444  0.476    3.10–3.20  +0.790  0.834  0.532  1.229  0.000  0.566    3.20–3.30  +0.082  0.275  0.504  1.196  0.083  0.551    3.30–3.40  +0.116  0.327  0.518  1.787  0.000  0.558    3.40–3.50  +0.006  0.151  0.465  0.050  0.503  0.504    3.50–3.60  +0.005  0.079  0.337  0.047  0.381  0.382    3.60–3.70  +0.006  0.077  0.345  0.048  0.375  0.390    3.70–3.80  +0.006  0.119  0.444  0.055  0.464  0.485    3.80–3.90  +0.026  0.103  0.316  0.061  0.321  0.380    3.90–4.00  −0.003  0.160  0.491  0.054  0.505  0.525  MLZ  0.00–0.10  −99.000  −99.000  −99.000  −99.000  −99.000  −99.000    0.10–0.20  +0.016  0.068  0.170  0.054  0.207  0.234    0.20–0.30  +0.013  0.072  0.256  0.055  0.297  0.287    0.30–0.40  +0.002  0.053  0.182  0.050  0.251  0.226    0.40–0.50  +0.011  0.070  0.194  0.064  0.210  0.241    0.50–0.60  −0.001  0.065  0.217  0.054  0.258  0.249    0.60–0.70  +0.000  0.038  0.117  0.035  0.207  0.148    0.70–0.80  −0.005  0.050  0.132  0.054  0.184  0.171    0.80–0.90  −0.009  0.053  0.089  0.054  0.142  0.148    0.90–1.00  −0.001  0.048  0.148  0.046  0.221  0.174    1.00–1.10  −0.019  0.075  0.205  0.095  0.135  0.229    1.10–1.20  −0.011  0.054  0.124  0.056  0.164  0.167    1.20–1.30  −0.010  0.060  0.093  0.058  0.128  0.152    1.30–1.40  −0.009  0.063  0.140  0.068  0.159  0.189    1.40–1.50  −0.000  0.082  0.319  0.128  0.177  0.305    1.50–1.60  −0.018  0.072  0.291  0.106  0.215  0.285    1.60–1.70  −0.032  0.115  0.335  0.152  0.087  0.338    1.70–1.80  +0.006  0.134  0.320  0.159  0.076  0.355    1.80–1.90  +0.024  0.143  0.315  0.151  0.106  0.357    1.90–2.00  +0.032  0.152  0.319  0.145  0.102  0.362    2.00–2.10  +0.013  0.139  0.275  0.129  0.097  0.334    2.10–2.20  +0.023  0.137  0.285  0.133  0.081  0.338    2.20–2.30  +0.034  0.155  0.316  0.144  0.090  0.371    2.30–2.40  +0.044  0.168  0.369  0.149  0.102  0.403    2.40–2.50  +0.073  0.174  0.433  0.165  0.131  0.430    2.50–2.60  +0.059  0.154  0.391  0.160  0.167  0.418    2.60–2.70  +0.031  0.134  0.371  0.145  0.226  0.402    2.70–2.80  +0.030  0.157  0.419  0.149  0.298  0.445    2.80–2.90  +0.019  0.155  0.429  0.110  0.368  0.456    2.90–3.00  +0.004  0.150  0.441  0.080  0.431  0.472    3.00–3.10  −0.013  0.144  0.450  0.064  0.460  0.494    3.10–3.20  −0.013  0.171  0.479  0.066  0.487  0.526    3.20–3.30  −0.003  0.145  0.449  0.061  0.465  0.488    3.30–3.40  −0.011  0.115  0.407  0.057  0.440  0.454    3.40–3.50  −0.008  0.092  0.355  0.048  0.389  0.400    3.50–3.60  −0.002  0.077  0.318  0.050  0.344  0.368    3.60–3.70  −0.001  0.080  0.326  0.048  0.360  0.371    3.70–3.80  −0.007  0.081  0.266  0.051  0.282  0.327    3.80–3.90  +0.015  0.101  0.293  0.064  0.309  0.355    3.90–4.00  −0.000  0.076  0.290  0.050  0.303  0.343  NNPZ  0.00–0.10  +0.009  0.042  0.067  0.037  0.113  0.109    0.10–0.20  −0.001  0.054  0.147  0.047  0.197  0.196    0.20–0.30  −0.000  0.061  0.207  0.060  0.252  0.242    0.30–0.40  −0.004  0.045  0.153  0.043  0.243  0.198    0.40–0.50  −0.002  0.061  0.199  0.053  0.245  0.237    0.50–0.60  +0.001  0.067  0.230  0.054  0.268  0.262    0.60–0.70  +0.001  0.042  0.169  0.036  0.257  0.196    0.70–0.80  +0.000  0.047  0.165  0.045  0.243  0.195    0.80–0.90  −0.010  0.052  0.123  0.054  0.180  0.170    0.90–1.00  −0.004  0.047  0.192  0.041  0.271  0.198    1.00–1.10  −0.021  0.069  0.222  0.097  0.137  0.230    1.10–1.20  −0.012  0.051  0.139  0.051  0.192  0.170    1.20–1.30  −0.004  0.050  0.092  0.050  0.146  0.141    1.30–1.40  −0.002  0.055  0.121  0.057  0.164  0.164    1.40–1.50  −0.004  0.057  0.198  0.059  0.243  0.216    1.50–1.60  −0.005  0.067  0.268  0.069  0.277  0.269    1.60–1.70  −0.018  0.107  0.302  0.142  0.124  0.317    1.70–1.80  −0.001  0.132  0.333  0.158  0.089  0.353    1.80–1.90  +0.008  0.137  0.311  0.145  0.125  0.350    1.90–2.00  +0.009  0.131  0.267  0.124  0.096  0.322    2.00–2.10  +0.013  0.124  0.251  0.118  0.110  0.312    2.10–2.20  +0.026  0.140  0.295  0.132  0.108  0.353    2.20–2.30  +0.016  0.146  0.321  0.138  0.130  0.362    2.30–2.40  +0.022  0.138  0.299  0.136  0.138  0.355    2.40–2.50  +0.036  0.141  0.382  0.149  0.163  0.393    2.50–2.60  +0.026  0.136  0.348  0.134  0.214  0.389    2.60–2.70  +0.019  0.119  0.347  0.118  0.254  0.389    2.70–2.80  −0.002  0.106  0.310  0.081  0.300  0.357    2.80–2.90  +0.001  0.104  0.277  0.077  0.272  0.348    2.90–3.00  +0.006  0.088  0.291  0.071  0.293  0.344    3.00–3.10  +0.003  0.101  0.327  0.066  0.336  0.381    3.10–3.20  −0.004  0.100  0.317  0.068  0.341  0.378    3.20–3.30  −0.006  0.086  0.326  0.055  0.355  0.377    3.30–3.40  −0.009  0.102  0.375  0.055  0.412  0.422    3.40–3.50  −0.005  0.068  0.322  0.041  0.380  0.365    3.50–3.60  +0.003  0.073  0.301  0.048  0.339  0.349    3.60–3.70  +0.007  0.070  0.253  0.050  0.289  0.309    3.70–3.80  +0.011  0.081  0.259  0.055  0.274  0.318    3.80–3.90  +0.013  0.070  0.230  0.050  0.246  0.289    3.90–4.00  +0.001  0.088  0.277  0.061  0.298  0.345  Code  zphot  Bias  σconv  foutlier, conv  σ  foutlier  ⟨L(Δz)⟩  DEmP  0.00–0.10  +0.014  0.049  0.213  0.031  0.260  0.221    0.10–0.20  +0.004  0.044  0.112  0.042  0.168  0.157    0.20–0.30  +0.004  0.049  0.192  0.046  0.269  0.219    0.30–0.40  +0.002  0.042  0.135  0.042  0.238  0.181    0.40–0.50  +0.003  0.057  0.170  0.053  0.220  0.213    0.50–0.60  +0.007  0.055  0.200  0.047  0.256  0.231    0.60–0.70  +0.004  0.037  0.132  0.033  0.228  0.163    0.70–0.80  +0.003  0.041  0.115  0.042  0.210  0.154    0.80–0.90  −0.006  0.047  0.100  0.049  0.171  0.149    0.90–1.00  −0.000  0.040  0.148  0.038  0.239  0.165    1.00–1.10  −0.009  0.067  0.213  0.090  0.162  0.225    1.10–1.20  −0.006  0.050  0.141  0.052  0.203  0.171    1.20–1.30  −0.001  0.054  0.115  0.055  0.164  0.160    1.30–1.40  −0.001  0.056  0.123  0.061  0.163  0.172    1.40–1.50  −0.006  0.057  0.202  0.062  0.239  0.218    1.50–1.60  −0.007  0.070  0.268  0.091  0.231  0.268    1.60–1.70  −0.010  0.106  0.301  0.142  0.115  0.316    1.70–1.80  +0.000  0.113  0.273  0.132  0.136  0.319    1.80–1.90  +0.011  0.138  0.291  0.140  0.119  0.345    1.90–2.00  +0.003  0.124  0.280  0.118  0.117  0.323    2.00–2.10  +0.017  0.143  0.308  0.130  0.134  0.357    2.10–2.20  +0.014  0.154  0.330  0.139  0.119  0.373    2.20–2.30  +0.019  0.149  0.342  0.141  0.141  0.380    2.30–2.40  +0.020  0.153  0.337  0.140  0.166  0.387    2.40–2.50  +0.021  0.149  0.370  0.150  0.186  0.402    2.50–2.60  +0.014  0.132  0.342  0.122  0.250  0.388    2.60–2.70  −0.002  0.130  0.356  0.101  0.314  0.407    2.70–2.80  +0.007  0.120  0.334  0.086  0.307  0.382    2.80–2.90  +0.001  0.108  0.327  0.073  0.328  0.386    2.90–3.00  −0.001  0.109  0.364  0.067  0.373  0.406    3.00–3.10  +0.004  0.114  0.374  0.065  0.382  0.430    3.10–3.20  −0.005  0.129  0.434  0.059  0.460  0.482    3.20–3.30  −0.002  0.135  0.442  0.059  0.464  0.483    3.30–3.40  −0.008  0.117  0.406  0.051  0.435  0.450    3.40–3.50  −0.006  0.094  0.337  0.053  0.372  0.391    3.50–3.60  +0.002  0.070  0.312  0.044  0.361  0.353    3.60–3.70  −0.002  0.073  0.305  0.046  0.331  0.356    3.70–3.80  +0.007  0.086  0.321  0.052  0.337  0.374    3.80–3.90  +0.016  0.101  0.344  0.057  0.357  0.401    3.90–4.00  +0.024  0.099  0.313  0.058  0.322  0.377  EPHOR  0.00–0.10  −0.045  0.092  0.408  0.338  0.003  0.420    0.10–0.20  +0.002  0.054  0.155  0.048  0.202  0.207    0.20–0.30  +0.002  0.052  0.190  0.049  0.268  0.219    0.30–0.40  −0.003  0.042  0.152  0.039  0.253  0.194    0.40–0.50  −0.000  0.052  0.181  0.046  0.248  0.216    0.50–0.60  +0.005  0.065  0.251  0.050  0.296  0.274    0.60–0.70  +0.000  0.040  0.161  0.034  0.254  0.188    0.70–0.80  −0.004  0.046  0.157  0.044  0.237  0.188    0.80–0.90  −0.012  0.047  0.101  0.046  0.176  0.149    0.90–1.00  −0.003  0.045  0.182  0.040  0.262  0.190    1.00–1.10  −0.017  0.064  0.215  0.090  0.159  0.223    1.10–1.20  −0.016  0.053  0.147  0.052  0.192  0.178    1.20–1.30  −0.006  0.049  0.083  0.048  0.139  0.132    1.30–1.40  −0.002  0.049  0.094  0.051  0.155  0.143    1.40–1.50  −0.004  0.052  0.172  0.051  0.237  0.193    1.50–1.60  −0.000  0.065  0.262  0.067  0.280  0.262    1.60–1.70  −0.007  0.109  0.316  0.152  0.111  0.327    1.70–1.80  −0.009  0.127  0.313  0.147  0.114  0.341    1.80–1.90  +0.002  0.119  0.249  0.122  0.112  0.304    1.90–2.00  +0.002  0.133  0.274  0.124  0.098  0.322    2.00–2.10  +0.035  0.142  0.294  0.133  0.100  0.344    2.10–2.20  +0.032  0.137  0.343  0.140  0.110  0.354    2.20–2.30  +0.034  0.155  0.338  0.148  0.120  0.385    2.30–2.40  +0.007  0.134  0.312  0.128  0.204  0.368    2.40–2.50  +0.021  0.142  0.349  0.138  0.201  0.388    2.50–2.60  +0.033  0.132  0.370  0.143  0.212  0.389    2.60–2.70  +0.039  0.121  0.338  0.127  0.221  0.377    2.70–2.80  +0.020  0.118  0.275  0.092  0.221  0.344    2.80–2.90  +0.010  0.095  0.281  0.076  0.270  0.343    2.90–3.00  +0.007  0.090  0.312  0.069  0.315  0.353    3.00–3.10  +0.011  0.126  0.368  0.080  0.354  0.423    3.10–3.20  +0.004  0.113  0.364  0.070  0.370  0.419    3.20–3.30  +0.005  0.125  0.406  0.069  0.409  0.453    3.30–3.40  −0.001  0.104  0.354  0.062  0.379  0.412    3.40–3.50  +0.004  0.099  0.364  0.054  0.405  0.410    3.50–3.60  +0.007  0.062  0.262  0.042  0.316  0.304    3.60–3.70  +0.004  0.072  0.204  0.053  0.230  0.276    3.70–3.80  +0.000  0.081  0.281  0.057  0.299  0.340    3.80–3.90  +0.016  0.091  0.274  0.058  0.289  0.342    3.90–4.00  −0.016  0.087  0.380  0.039  0.441  0.419  EPHOR_AB  0.00–0.10  +0.001  0.068  0.218  0.054  0.248  0.254    0.10–0.20  −0.005  0.044  0.125  0.040  0.185  0.168    0.20–0.30  −0.004  0.051  0.190  0.048  0.266  0.221    0.30–0.40  −0.002  0.034  0.106  0.032  0.206  0.140    0.40–0.50  +0.004  0.058  0.187  0.052  0.232  0.226    0.50–0.60  +0.001  0.057  0.198  0.048  0.254  0.229    0.60–0.70  −0.003  0.036  0.105  0.033  0.202  0.139    0.70–0.80  +0.001  0.038  0.076  0.041  0.166  0.124    0.80–0.90  −0.005  0.045  0.080  0.044  0.152  0.129    0.90–1.00  −0.003  0.043  0.144  0.041  0.225  0.166    1.00–1.10  −0.019  0.075  0.237  0.106  0.108  0.242    1.10–1.20  −0.006  0.051  0.135  0.053  0.194  0.167    1.20–1.30  −0.004  0.055  0.101  0.053  0.146  0.149    1.30–1.40  −0.007  0.056  0.115  0.058  0.159  0.164    1.40–1.50  −0.012  0.057  0.190  0.054  0.255  0.209    1.50–1.60  −0.005  0.064  0.225  0.078  0.214  0.236    1.60–1.70  −0.010  0.098  0.266  0.128  0.132  0.288    1.70–1.80  −0.013  0.111  0.250  0.126  0.104  0.297    1.80–1.90  −0.011  0.118  0.232  0.118  0.091  0.293    1.90–2.00  −0.009  0.118  0.250  0.112  0.092  0.296    2.00–2.10  −0.011  0.118  0.211  0.110  0.068  0.281    2.10–2.20  −0.005  0.114  0.208  0.106  0.095  0.282    2.20–2.30  +0.002  0.129  0.256  0.124  0.119  0.319    2.30–2.40  +0.008  0.124  0.270  0.134  0.110  0.329    2.40–2.50  +0.014  0.124  0.290  0.134  0.176  0.348    2.50–2.60  −0.013  0.113  0.279  0.099  0.252  0.344    2.60–2.70  −0.003  0.114  0.308  0.088  0.287  0.370    2.70–2.80  −0.006  0.111  0.329  0.072  0.336  0.384    2.80–2.90  −0.009  0.111  0.363  0.067  0.390  0.414    2.90–3.00  −0.010  0.109  0.369  0.064  0.383  0.416    3.00–3.10  −0.007  0.103  0.359  0.058  0.384  0.418    3.10–3.20  −0.001  0.109  0.405  0.057  0.431  0.448    3.20–3.30  −0.008  0.100  0.351  0.058  0.385  0.406    3.30–3.40  −0.005  0.078  0.312  0.050  0.360  0.364    3.40–3.50  −0.003  0.066  0.274  0.046  0.308  0.322    3.50–3.60  −0.005  0.057  0.230  0.043  0.280  0.278    3.60–3.70  −0.010  0.074  0.232  0.050  0.255  0.293    3.70–3.80  −0.003  0.080  0.291  0.053  0.318  0.339    3.80–3.90  −0.005  0.095  0.303  0.058  0.320  0.368    3.90–4.00  −0.008  0.080  0.209  0.054  0.215  0.283  FRANKEN-Z  0.00–0.10  +0.011  0.044  0.281  0.021  0.402  0.282    0.10–0.20  −0.006  0.045  0.144  0.039  0.207  0.182    0.20–0.30  −0.003  0.044  0.198  0.036  0.303  0.225    0.30–0.40  −0.002  0.035  0.119  0.033  0.233  0.157    0.40–0.50  −0.001  0.053  0.162  0.048  0.230  0.204    0.50–0.60  +0.002  0.052  0.193  0.045  0.261  0.223    0.60–0.70  −0.001  0.036  0.125  0.033  0.225  0.156    0.70–0.80  −0.002  0.039  0.101  0.040  0.196  0.140    0.80–0.90  −0.008  0.044  0.086  0.044  0.169  0.135    0.90–1.00  −0.002  0.040  0.147  0.037  0.236  0.162    1.00–1.10  −0.008  0.060  0.198  0.074  0.195  0.211    1.10–1.20  −0.007  0.045  0.119  0.045  0.198  0.153    1.20–1.30  −0.003  0.049  0.091  0.049  0.151  0.139    1.30–1.40  −0.004  0.050  0.109  0.052  0.175  0.153    1.40–1.50  −0.006  0.051  0.191  0.050  0.262  0.206    1.50–1.60  −0.005  0.062  0.245  0.062  0.275  0.251    1.60–1.70  −0.012  0.095  0.268  0.122  0.145  0.288    1.70–1.80  +0.008  0.105  0.237  0.120  0.139  0.295    1.80–1.90  +0.000  0.116  0.249  0.122  0.116  0.302    1.90–2.00  −0.005  0.117  0.257  0.117  0.101  0.301    2.00–2.10  +0.002  0.130  0.276  0.120  0.115  0.321    2.10–2.20  +0.014  0.129  0.278  0.121  0.115  0.329    2.20–2.30  +0.018  0.147  0.291  0.138  0.119  0.353    2.30–2.40  +0.024  0.147  0.326  0.146  0.144  0.378    2.40–2.50  +0.019  0.121  0.334  0.132  0.203  0.356    2.50–2.60  +0.016  0.128  0.332  0.132  0.239  0.380    2.60–2.70  +0.005  0.111  0.297  0.093  0.270  0.354    2.70–2.80  −0.001  0.098  0.310  0.075  0.310  0.356    2.80–2.90  +0.001  0.105  0.328  0.070  0.338  0.377    2.90–3.00  +0.007  0.105  0.324  0.072  0.324  0.378    3.00–3.10  +0.004  0.096  0.340  0.059  0.354  0.393    3.10–3.20  +0.007  0.098  0.364  0.055  0.384  0.413    3.20–3.30  +0.006  0.116  0.421  0.053  0.444  0.465    3.30–3.40  +0.001  0.092  0.387  0.047  0.430  0.421    3.40–3.50  −0.001  0.078  0.299  0.049  0.348  0.350    3.50–3.60  +0.004  0.064  0.277  0.042  0.324  0.319    3.60–3.70  +0.007  0.065  0.237  0.046  0.273  0.293    3.70–3.80  +0.003  0.079  0.272  0.052  0.295  0.323    3.80–3.90  +0.013  0.092  0.334  0.050  0.344  0.376    3.90–4.00  +0.010  0.084  0.294  0.054  0.313  0.348  Mizuki  0.00–0.10  −0.272  0.286  0.550  0.330  0.000  0.522    0.10–0.20  +0.001  0.070  0.283  0.048  0.317  0.301    0.20–0.30  +0.007  0.059  0.193  0.055  0.257  0.231    0.30–0.40  +0.005  0.042  0.132  0.039  0.224  0.173    0.40–0.50  +0.013  0.074  0.244  0.061  0.270  0.287    0.50–0.60  +0.016  0.081  0.258  0.073  0.246  0.290    0.60–0.70  −0.001  0.047  0.137  0.044  0.218  0.177    0.70–0.80  −0.010  0.053  0.088  0.054  0.138  0.152    0.80–0.90  −0.012  0.053  0.095  0.050  0.161  0.151    0.90–1.00  −0.002  0.055  0.144  0.055  0.192  0.184    1.00–1.10  −0.028  0.088  0.239  0.111  0.091  0.259    1.10–1.20  −0.019  0.064  0.146  0.076  0.134  0.195    1.20–1.30  −0.003  0.061  0.101  0.062  0.137  0.162    1.30–1.40  −0.003  0.064  0.156  0.067  0.179  0.204    1.40–1.50  −0.006  0.064  0.228  0.069  0.244  0.242    1.50–1.60  +0.000  0.074  0.304  0.107  0.243  0.298    1.60–1.70  +0.012  0.113  0.329  0.153  0.101  0.332    1.70–1.80  +0.043  0.161  0.369  0.175  0.077  0.394    1.80–1.90  +0.033  0.169  0.385  0.172  0.080  0.398    1.90–2.00  +0.017  0.160  0.361  0.160  0.117  0.383    2.00–2.10  +0.019  0.161  0.341  0.149  0.102  0.376    2.10–2.20  +0.009  0.148  0.297  0.139  0.108  0.361    2.20–2.30  +0.024  0.166  0.347  0.153  0.128  0.404    2.30–2.40  +0.028  0.166  0.369  0.161  0.157  0.420    2.40–2.50  +0.011  0.146  0.362  0.148  0.199  0.405    2.50–2.60  +0.029  0.143  0.389  0.150  0.213  0.413    2.60–2.70  +0.019  0.142  0.385  0.132  0.285  0.426    2.70–2.80  +0.000  0.141  0.390  0.101  0.355  0.433    2.80–2.90  −0.004  0.165  0.445  0.081  0.439  0.491    2.90–3.00  +0.006  0.181  0.483  0.076  0.479  0.508    3.00–3.10  +0.006  0.130  0.435  0.062  0.444  0.476    3.10–3.20  +0.790  0.834  0.532  1.229  0.000  0.566    3.20–3.30  +0.082  0.275  0.504  1.196  0.083  0.551    3.30–3.40  +0.116  0.327  0.518  1.787  0.000  0.558    3.40–3.50  +0.006  0.151  0.465  0.050  0.503  0.504    3.50–3.60  +0.005  0.079  0.337  0.047  0.381  0.382    3.60–3.70  +0.006  0.077  0.345  0.048  0.375  0.390    3.70–3.80  +0.006  0.119  0.444  0.055  0.464  0.485    3.80–3.90  +0.026  0.103  0.316  0.061  0.321  0.380    3.90–4.00  −0.003  0.160  0.491  0.054  0.505  0.525  MLZ  0.00–0.10  −99.000  −99.000  −99.000  −99.000  −99.000  −99.000    0.10–0.20  +0.016  0.068  0.170  0.054  0.207  0.234    0.20–0.30  +0.013  0.072  0.256  0.055  0.297  0.287    0.30–0.40  +0.002  0.053  0.182  0.050  0.251  0.226    0.40–0.50  +0.011  0.070  0.194  0.064  0.210  0.241    0.50–0.60  −0.001  0.065  0.217  0.054  0.258  0.249    0.60–0.70  +0.000  0.038  0.117  0.035  0.207  0.148    0.70–0.80  −0.005  0.050  0.132  0.054  0.184  0.171    0.80–0.90  −0.009  0.053  0.089  0.054  0.142  0.148    0.90–1.00  −0.001  0.048  0.148  0.046  0.221  0.174    1.00–1.10  −0.019  0.075  0.205  0.095  0.135  0.229    1.10–1.20  −0.011  0.054  0.124  0.056  0.164  0.167    1.20–1.30  −0.010  0.060  0.093  0.058  0.128  0.152    1.30–1.40  −0.009  0.063  0.140  0.068  0.159  0.189    1.40–1.50  −0.000  0.082  0.319  0.128  0.177  0.305    1.50–1.60  −0.018  0.072  0.291  0.106  0.215  0.285    1.60–1.70  −0.032  0.115  0.335  0.152  0.087  0.338    1.70–1.80  +0.006  0.134  0.320  0.159  0.076  0.355    1.80–1.90  +0.024  0.143  0.315  0.151  0.106  0.357    1.90–2.00  +0.032  0.152  0.319  0.145  0.102  0.362    2.00–2.10  +0.013  0.139  0.275  0.129  0.097  0.334    2.10–2.20  +0.023  0.137  0.285  0.133  0.081  0.338    2.20–2.30  +0.034  0.155  0.316  0.144  0.090  0.371    2.30–2.40  +0.044  0.168  0.369  0.149  0.102  0.403    2.40–2.50  +0.073  0.174  0.433  0.165  0.131  0.430    2.50–2.60  +0.059  0.154  0.391  0.160  0.167  0.418    2.60–2.70  +0.031  0.134  0.371  0.145  0.226  0.402    2.70–2.80  +0.030  0.157  0.419  0.149  0.298  0.445    2.80–2.90  +0.019  0.155  0.429  0.110  0.368  0.456    2.90–3.00  +0.004  0.150  0.441  0.080  0.431  0.472    3.00–3.10  −0.013  0.144  0.450  0.064  0.460  0.494    3.10–3.20  −0.013  0.171  0.479  0.066  0.487  0.526    3.20–3.30  −0.003  0.145  0.449  0.061  0.465  0.488    3.30–3.40  −0.011  0.115  0.407  0.057  0.440  0.454    3.40–3.50  −0.008  0.092  0.355  0.048  0.389  0.400    3.50–3.60  −0.002  0.077  0.318  0.050  0.344  0.368    3.60–3.70  −0.001  0.080  0.326  0.048  0.360  0.371    3.70–3.80  −0.007  0.081  0.266  0.051  0.282  0.327    3.80–3.90  +0.015  0.101  0.293  0.064  0.309  0.355    3.90–4.00  −0.000  0.076  0.290  0.050  0.303  0.343  NNPZ  0.00–0.10  +0.009  0.042  0.067  0.037  0.113  0.109    0.10–0.20  −0.001  0.054  0.147  0.047  0.197  0.196    0.20–0.30  −0.000  0.061  0.207  0.060  0.252  0.242    0.30–0.40  −0.004  0.045  0.153  0.043  0.243  0.198    0.40–0.50  −0.002  0.061  0.199  0.053  0.245  0.237    0.50–0.60  +0.001  0.067  0.230  0.054  0.268  0.262    0.60–0.70  +0.001  0.042  0.169  0.036  0.257  0.196    0.70–0.80  +0.000  0.047  0.165  0.045  0.243  0.195    0.80–0.90  −0.010  0.052  0.123  0.054  0.180  0.170    0.90–1.00  −0.004  0.047  0.192  0.041  0.271  0.198    1.00–1.10  −0.021  0.069  0.222  0.097  0.137  0.230    1.10–1.20  −0.012  0.051  0.139  0.051  0.192  0.170    1.20–1.30  −0.004  0.050  0.092  0.050  0.146  0.141    1.30–1.40  −0.002  0.055  0.121  0.057  0.164  0.164    1.40–1.50  −0.004  0.057  0.198  0.059  0.243  0.216    1.50–1.60  −0.005  0.067  0.268  0.069  0.277  0.269    1.60–1.70  −0.018  0.107  0.302  0.142  0.124  0.317    1.70–1.80  −0.001  0.132  0.333  0.158  0.089  0.353    1.80–1.90  +0.008  0.137  0.311  0.145  0.125  0.350    1.90–2.00  +0.009  0.131  0.267  0.124  0.096  0.322    2.00–2.10  +0.013  0.124  0.251  0.118  0.110  0.312    2.10–2.20  +0.026  0.140  0.295  0.132  0.108  0.353    2.20–2.30  +0.016  0.146  0.321  0.138  0.130  0.362    2.30–2.40  +0.022  0.138  0.299  0.136  0.138  0.355    2.40–2.50  +0.036  0.141  0.382  0.149  0.163  0.393    2.50–2.60  +0.026  0.136  0.348  0.134  0.214  0.389    2.60–2.70  +0.019  0.119  0.347  0.118  0.254  0.389    2.70–2.80  −0.002  0.106  0.310  0.081  0.300  0.357    2.80–2.90  +0.001  0.104  0.277  0.077  0.272  0.348    2.90–3.00  +0.006  0.088  0.291  0.071  0.293  0.344    3.00–3.10  +0.003  0.101  0.327  0.066  0.336  0.381    3.10–3.20  −0.004  0.100  0.317  0.068  0.341  0.378    3.20–3.30  −0.006  0.086  0.326  0.055  0.355  0.377    3.30–3.40  −0.009  0.102  0.375  0.055  0.412  0.422    3.40–3.50  −0.005  0.068  0.322  0.041  0.380  0.365    3.50–3.60  +0.003  0.073  0.301  0.048  0.339  0.349    3.60–3.70  +0.007  0.070  0.253  0.050  0.289  0.309    3.70–3.80  +0.011  0.081  0.259  0.055  0.274  0.318    3.80–3.90  +0.013  0.070  0.230  0.050  0.246  0.289    3.90–4.00  +0.001  0.088  0.277  0.061  0.298  0.345  *The number are for all galaxies down to i = 25. View Large 5.3 Code–code comparisons In the previous subsection, we plotted the three metrics (bias, dispersion, and outlier rate) separately for each code. It is useful, however, to use a single metric to compare the performance between the codes. For this, we use the loss parameter, L(Δz), introduced earlier. Because this is not a popular statistic used in the literature, we first show its relationship to the other statistical measures in figure 5; this shows MLZ, but the other codes behave similarly. While all of the bias, scatter, and outlier rate are correlated (all of them get worse at fainter magnitudes), it is clear that the mean loss most strongly correlates with the outlier rate. Loss should also change with bias and scatter at a fixed outlier rate by definition, but it is the outlier rate that increases drastically at faint magnitudes, and the mean loss most strongly correlates with that parameter. Fig. 5. View largeDownload slide Relationship between loss and other metrics. The symbols are color-coded according to the i-band magnitude cut applied. This is for MLZ using the Wide-depth median seeing catalog, but the other codes show similar trends. (Color online) Fig. 5. View largeDownload slide Relationship between loss and other metrics. The symbols are color-coded according to the i-band magnitude cut applied. This is for MLZ using the Wide-depth median seeing catalog, but the other codes show similar trends. (Color online) Figures 6 and 7 show the mean loss as a function of magnitude and zphot, respectively. All the codes show similar behavior in these figures: the accuracy starts to get worse around i = 23, and the redshift range of 0.2 ≲ z ≲ 1.5 shows the best performance. Mizuki tends to perform worse than the other codes. Although it was trained on an earlier version of the training sample with suboptimal weights (see subsection 3.6), it is the only classical template-fitting code and it might suggest that machine-learning codes outperform template fitting. There are advantages and disadvantages in both techniques and we will discuss them further in section 8. Again, note that the metrics for FRANKEN-Z are likely to be overly optimistic, given some degree of over-fitting. Fig. 6. View largeDownload slide Mean loss as a function of i-band magnitude for all the codes. The symbols are explained in the figure. (Color online) Fig. 6. View largeDownload slide Mean loss as a function of i-band magnitude for all the codes. The symbols are explained in the figure. (Color online) Fig. 7. View largeDownload slide As figure 6, but as a function of zphot. (Color online) Fig. 7. View largeDownload slide As figure 6, but as a function of zphot. (Color online) 5.4 Seeing and depth dependence Photometric accuracy is not only a function of integration time and sky transparency, but also seeing. As described in subsection 2.3, we have generated the COSMOS Wide-depth stacks for three different seeing FWHMs. We use them to evaluate the seeing dependence of our photo-z accuracy at the Wide depth. Figure 8 shows ⟨L(Δz)⟩ as a function of seeing. Loss is larger at worse seeing, as expected, and we find that Δ⟨L(Δz)⟩ ∼ 0.05 between the two extremes. Most of the HSC data are taken under 0$${^{\prime\prime}_{.}}$$5–1$${^{\prime\prime}_{.}}$$0 seeing (Aihara et al. 2018b), and figure 8 gives the peak–peak variation of our photo-z performance across the Wide survey. EPHOR delivers photo-z’s computed with CModel and PSF-matched aperture photometry (EPHOR and EPHOR_AB, respectively). A comparison between them shows how strongly each photometry technique suffers from the seeing variation. The PSF-matched photometry turns out to be less strongly affected by seeing than CModel: Δ⟨L(Δz)⟩ ∼ 0.03 and 0.06 for PSF-matched and CModel photometry, respectively. The weaker seeing dependence of the PSF-matched photometry is not surprising because the images are smoothed to 1$${^{\prime\prime}_{.}}$$1 FWHM, regardless of the native seeing. It is, however, rather surprising that the measurements under the native seeing deliver poorer photo-z accuracy. But, we note that the current CModel has issues with a prior, which affects the resultant photometry (Bosch et al. 2018; Huang et al. 2018). It is unlikely that the color measurements are severely affected, but fluxes are undoubtedly affected. Also, the deblending algorithm tends to fail in dense regions such as cluster cores (Aihara et al. 2018b), which also affects CModel measurements. The PSF-matched photometry suffers less from the deblending issue because it is performed without deblending. Future improvements in the measurement algorithms will make CModel work better. Fig. 8. View largeDownload slide Mean loss as a function of seeing. (Color online) Fig. 8. View largeDownload slide Mean loss as a function of seeing. (Color online) The depth dependence is shown in figure 9. Again, all the codes behave similarly and the mean loss is smaller by ∼0.1 at the UltraDeep depth. Although not shown in the figure, the improvement is not limited to 0.2 < z < 1.5, but is observed at all redshifts. This implies that obtaining photometry in more filters is not the only way to improve photo-z’s. Going deeper can be a useful alternative. Fig. 9. View largeDownload slide As figure 8, but as a function of depth. (Color online) Fig. 9. View largeDownload slide As figure 8, but as a function of depth. (Color online) 5.5 Cut on the risk parameter We have characterized our photo-z performance using all galaxies down to i = 25 without any clipping of potential outliers. We can achieve reasonably good photo-z accuracy at a somewhat limited redshift range due to the filter set, as discussed above. Also, our photo-z’s are of course not perfect, and there are always outliers even within the good redshift range. There are a few quantities that can be used to indicate the reliability of photo-z, such as C(z) and odds (Benitez 2000), which allow us to remove potential outliers. We have introduced a new parameter, R(z), in subsection 4.2, and here we compare this new parameter with the commonly used C(z). Figure 10 compares C(zphot) and R(zphot). As defined earlier, zphot is the best point estimate. We remove objects with C(zphot)/R(zphot) smaller/larger than a threshold value and plot the resultant ⟨L(Δz)⟩ as a function of the fraction of objects removed. At a given fraction of removed objects, ⟨L(Δz)⟩ is always smaller for R(zphot) than for C(zphot). For instance, at fremoved = 0.5 (i.e., we remove half of the objects), which roughly corresponds to C(zphot) < 0.5 and R(zphot) > 0.9 cuts, loss is smaller for R(zphot) by about 0.02. R(zphot) is designed to minimize loss, and thus this may not be a fair comparison, but we observe the same trend if we plot other quantities such as the outlier rate. This demonstrates that R(z) works better at identifying outliers than the commonly used C(z). Fig. 10. View largeDownload slide Loss plotted against the fraction of objects removed by applying a cut on C(zphot) and R(zphot). zphot is denoted as z in the figure for simplicity. The dashed and solid curves are for C(zphot) and R(zphot), and the threshold applied for each of them is shown in the figure. This is for Mizuki, but the other codes show a similar trend. (Color online) Fig. 10. View largeDownload slide Loss plotted against the fraction of objects removed by applying a cut on C(zphot) and R(zphot). zphot is denoted as z in the figure for simplicity. The dashed and solid curves are for C(zphot) and R(zphot), and the threshold applied for each of them is shown in the figure. This is for Mizuki, but the other codes show a similar trend. (Color online) 6 Accuracy of the PDF We focused on the point statistics in the previous section. We now move on to discuss the accuracy of the full PDF. We first focus on the N(z) distribution of galaxies and then turn our attention to the probability integral transform and continuous ranked probability score to evaluate the PDF accuracy. 6.1 N(z) distribution In various scientific uses, we often consider not only the redshift for a single galaxy but also the global properties averaged over a number of objects. In this section, we show the redshift distributions of a photometric sample from the S16A internal release and compare them for the seven different photo-z codes. 6.1.1 Internal comparisons As we will discuss in section 7, we randomly draw a redshift from P(z) for each object (zMC). We first demonstrate that this Monte Carlo draw from the PDF reproduces the original PDF well, and is a very useful point estimate for a statistical sample. In figure 11, we compare the stacked PDF and the sum of zMC using a Gaussian kernel density estimator (KDE),   \begin{equation} N^{\rm P}(z) = \frac{1}{n} \sum _i^{n} P_i(z) \end{equation} (16)  \begin{equation} N^{\rm MC}(z) = \frac{1}{\sqrt{2\pi }nh} \sum _i^n \exp \left[ \frac{(z-z_{{\rm MC}, i})^2}{2h^2} \right], \end{equation} (17)where the kernel width h is set to the PDF resolution, 0.05 for EPHOR and EPHOR_AB and 0.01 for all the other codes. The estimator reduces the discreteness of the sample, but we found that given the large number of objects, we do not see any major differences between the classical count-up histogram and KDE. Fig. 11. View largeDownload slide N(z) distributions for all galaxies in the Wide layer inferred using a few different estimators: sum of full PDF (gray histogram), and Gaussian KDE for zMC and zbest (see figure legend). The sum of full PDF and N(zMC) agree very well, while the N(zbest) estimates show sharp redshift spikes. This is likely to be due to the spikes present in the training data from COSMOS. Fig. 11. View largeDownload slide N(z) distributions for all galaxies in the Wide layer inferred using a few different estimators: sum of full PDF (gray histogram), and Gaussian KDE for zMC and zbest (see figure legend). The sum of full PDF and N(zMC) agree very well, while the N(zbest) estimates show sharp redshift spikes. This is likely to be due to the spikes present in the training data from COSMOS. As shown in the figure, we see a good agreement between NMC and NP for most codes, although NMC fails to trace the small-scale spiky features in NNPZ, EPHOR, and EPHOR_AB seen in NP. In the same figure, we also plot the N(z) distribution from zbest using equation (17). Although the zbest is the optimal point estimate in terms of minimizing the risk function (see subsection 4.2), Nbest amplifies the wiggle feature of the N(z) distribution. This might imply that the point estimates are affected by inhomogeneities in the training sample. For instance, the local peaks around z ∼ 1.5 are a consequence of the bumpy structure in the COSMOS 30 band photo-z, on which we rely highly to calibrate our photo-z’s (see also figure 13). Nbest for Mizuki is least affected since the template fitting does not rely on the training sample very much, while machine-learning codes do. In the following, we use NMC as representative of the redshift distribution instead of NP, since summing the full PDF is computationally much more expensive. Figure 12 shows the N(z) distribution from zMC for bright (i < 22.5) and faint (i > 22.5) samples. The sharp drop of the bright sample at z ∼ 1 reflects that we have few bright objects at z > 1. On the other hand, we have galaxies out to z ∼ 6 in the fainter sample. Although there are subtle differences between the codes, the overall redshift distributions are similar for all of the codes, which is encouraging. Fig. 12. View largeDownload slide N(z) distribution for bright (red) and faint (blue) samples. The median distribution of each photo-z code is also shown for bright (magenta thin line) and faint (cyan thin line) samples. EPHOR looks smoother than the others due to a larger redshift bin size. Fig. 12. View largeDownload slide N(z) distribution for bright (red) and faint (blue) samples. The median distribution of each photo-z code is also shown for bright (magenta thin line) and faint (cyan thin line) samples. EPHOR looks smoother than the others due to a larger redshift bin size. 6.1.2 External comparisons We have compared the internal consistency in the previous section. We now turn our attention to external comparisons using the reference redshifts in the COSMOS Wide-depth median stack. Figure 13 shows a comparison between NMC and N(z) based on the reference redshifts. Assuming that the reference redshifts are correct, deviations from N(z) are an indication of an incorrect PDF. While all the codes reproduce the overall N(z) reasonably well, NNPZ reproduces N(z) most accurately. Mizuki misses a peak at z ∼ 0.35, and EPHOR, MLZ, and NNPZ tend to overestimate at z ∼ 0.7. This has implications for weak-lensing science, which often relies on N(z) from photo-z. However, detailed discussions of the over-/underestimated N(z) for weak-lensing are beyond the scope of the paper and can be found elsewhere (S. More et al. in preparation). Fig. 13. View largeDownload slide N(z) distributions from COSMOS Wide-depth stacked images with median seeing (solid line) and reference redshifts (gray shaded histogram). (Color online) Fig. 13. View largeDownload slide N(z) distributions from COSMOS Wide-depth stacked images with median seeing (solid line) and reference redshifts (gray shaded histogram). (Color online) We have trained our codes using galaxies that are primarily from COSMOS, especially at faint magnitudes, and we have compared our N(z) against COSMOS. Reweighting the training galaxies to reproduce the HSC Wide sample largely eliminates the circularity here. However, it will certainly be useful to have a separate field with different N(z) for more comparisons. Such a COSMOS-like field with accurate photo-z’s down to faint magnitudes is currently not available, which is a one of the major limitations of our photo-z tests. We will discuss our future directions in section 8. 6.2 Tests on the PDF As a further test of the accuracy of the PDF, we apply two techniques; probability integral transform (PIT) and continuous ranked probability score (CRPS). These are summarized in Polsterer, D’Isanto, and Gieseke (2016), but a brief description is given here. PIT was proposed as a visual diagnostic tool to check the calibration of the PDF. It is a very simple diagnostic and one only needs to draw a histogram of the integrated probability,   \begin{equation} \mathrm{PIT}(z_{\rm ref})=\int _0^{z_{\rm ref}} P(z) dz, \end{equation} (18)for all objects in the test sample. The left panels in figure 14 show the PIT histograms for all the codes. If the PDF is calibrated well, we expect to observe a flat PIT distribution. Deviations from a flat distribution are an indication of an incorrect PDF, and this formed a basis of the empirical PDF recalibration by Bordoloi, Lilly, and Amara (2010). EPHOR_AB shows a convex shape, which is a clear indication of an overdispersed PDF, i.e., the PDF is too wide. On the other hand, Mizuki has a concave shape, indicating that the PDFs are underdispersed, i.e., the PDF is too narrow. Most of the other codes show a relatively flat distribution, except at the two extremes of the distribution, where many codes show a spike. These spikes are caused by outliers, and the figures suggest that the outliers are not properly captured in the PDFs. Fig. 14. View largeDownload slide PIT (left) and CRPS (right) for all the codes. The horizontal line in the left panel is just to guide the eye. (Color online) Fig. 14. View largeDownload slide PIT (left) and CRPS (right) for all the codes. The horizontal line in the left panel is just to guide the eye. (Color online) FRANKEN-Z shows an interesting PIT distribution with a peak at the center. The peak indicates that a larger than expected fraction of objects have the median redshift almost exactly at zref, which suggests that the PDF is too accurate. We do not expect to see such a feature in the presence of random uncertainties. While we have not fully understood the origin of the peak, we tentatively interpret it as a sign of over-fitting. Most likely, this peak is due to FRANKEN-Z’s inclusion of both the training and target errors when deriving likelihoods. Unlike other nearest-neighbor methods such as NNPZ, which select neighbors and derive weights using Monte Carlo procedures based on (modifications to the) Euclidean norm, FRANKEN-Z computes the intrinsic likelihood expected if training/testing objects were Monte Carlo realizations of the same underlying galaxy (J. Speagle et al. in preparation). Objects whose photometry between the Wide-depth stacks and Deep/UltraDeep observations are not fully independent can thus sometimes deviate much less than expected, leading to large contributions to the posterior and subsequent signs of over-fitting. We note that numerous cross-validation and hold-out tests have not found evidence of such behavior in the native training sample. All of the codes have some degree of deviation from the flat PIT distribution. This motivates us to use the PIT distribution to empirically recalibrate our P(z) (Bordoloi et al. 2010) in our future releases, as it is likely to improve our overall performance. We turn our attention to the other technique, CRPS, which is a measure of the “distance” between the PDF and zref, and is defined as   \begin{equation} \mathrm{CRPS} =\int _{-\infty }^{+\infty } \lbrace \mathrm{PIT}(z) - H(z-z_{\rm ref})\rbrace ^2 dz, \end{equation} (19)where H(z − zref) the Heaviside step function:   \begin{equation} H(x)=\left\lbrace \begin{array}{l}0\ \mathrm{if}\ x<0, \\ 1\ \mathrm{if}\ x\ge 0. \end{array}\right. \end{equation} (20)The right panels of figure 14 show CRPS for all the codes. When PDFs are calibrated well, the mean CRPS is small. A large CRPS is an indication of an incorrect PDF. To the first order, all the codes perform similarly well: ⟨log CRPS⟩ ∼ −1. However, there are small differences in CRPS between the codes, and machine-learning codes once again tend to perform better than the classical template-fitting code. It is interesting to note that a code with good performance with point estimates does not necessarily give a small CRPS. For instance, EPHOR_AB has a smaller loss than EPHOR, as shown in figure 8. However, the CRPS in figure 14 is larger, suggesting that the PDF is less accurate. The PIT distribution indicates that EPHOR has over-dispersed PDFs, which are likely to be driving the slightly larger CRPS. The analysis here suggests that accurate point estimates do not necessarily mean that the PDFs are accurate. They are obviously closely related to each other, but not exactly the same. Thus, in order to evaluate photo-z performance, one needs to look at both the point estimates and the PDFs. 7 Data products We make our photo-z products available to the community. This section summarizes our target selection criteria, “common” outputs that are available for all the codes, and code-specific outputs. HSC-SSP PDR1 includes our photo-z’s for the Deep and UltraDeep layers, covering over 30 square degrees in total. Due to a technical issue during the photo-z production run, we were unable to include our photo-z’s for the Wide layer in PDR1, but they were made public as part of the first incremental data release, which occurred in 2017 June. It is important to note that each code applies various cuts to select objects for photo-z production. That is, each code is applied to a different set of objects (but with a significant overlap) due to features of the code. Table 4 summarizes the target selection by codes. The table also indicates whether there are additional outputs from the code, which we will elaborate below. Most codes imposed detect_is_primary to select primary objects, except for EPHOR. DEmP, and MLZ compute photo-z’s for all the primary objects, but FRANKEN-Z and NNPZ require good photometry in all the bands in addition to the primary flag. Mizuki computes photo-z’s for primary objects with good CModel photometry in at least three bands (inclusive). Table 4. Target selection applied by each code. Code  Target selection  Number of objects  Other quantities  DEmP  detect_is_primary is True  171721095  None  EPHOR  Objects with CModel fluxes in all five bands  197227501  None  EPHOR_AB  Objects with afterburner fluxes in all five bands  221617662  None  FRANKEN-Z  detect_is_primary is True  135966862  Many    Objects with afterburner fluxes in all five bands      Mizuki  detect_is_primary is True  144107354  Many    Objects with CModel fluxes in at least three bands      MLZ  detect_is_primary is True  171721095  Flux flag  NNPZ  detect_is_primary is True  163627623  Neighbor redshifts and weights    Objects with CModel fluxes in all five bands      Code  Target selection  Number of objects  Other quantities  DEmP  detect_is_primary is True  171721095  None  EPHOR  Objects with CModel fluxes in all five bands  197227501  None  EPHOR_AB  Objects with afterburner fluxes in all five bands  221617662  None  FRANKEN-Z  detect_is_primary is True  135966862  Many    Objects with afterburner fluxes in all five bands      Mizuki  detect_is_primary is True  144107354  Many    Objects with CModel fluxes in at least three bands      MLZ  detect_is_primary is True  171721095  Flux flag  NNPZ  detect_is_primary is True  163627623  Neighbor redshifts and weights    Objects with CModel fluxes in all five bands      *The number of objects that satisfy the selection is shown. Details of other quantities available in the catalog can be found in section 7. View Large All the codes generate a PDF for each object. We run a common script to compute various point estimates, confidence intervals, and other useful statistics. The common outputs are summarized in table 5. In addition to these common outputs, there are the following code-specific outputs: Table 5. Common photo-z parameters available for all the codes. Key  Description  object_id  Unique object id to be used to join with the photometry tables.  photoz_X  Photo-z point estimate, where X is either mean, mode, median, or best.  photoz_mc  Monte Carlo draw from the full PDF.  photoz_conf_X  Photo-z confidence value defined by equation (15) at photoz_X.  photoz_risk_X  Risk parameter defined by equation (13) at photoz_X.  photoz_std_X  Second-order moment around a point estimate (photoz_X) derived from full PDF.  photoz_err68_min  16th percentile in the PDF.  photoz_err68_max  84th percentile in the PDF.  photoz_err95_min  2.5th percentile in the PDF.  photoz_err95_max  97.5th percentile in the PDF.  Key  Description  object_id  Unique object id to be used to join with the photometry tables.  photoz_X  Photo-z point estimate, where X is either mean, mode, median, or best.  photoz_mc  Monte Carlo draw from the full PDF.  photoz_conf_X  Photo-z confidence value defined by equation (15) at photoz_X.  photoz_risk_X  Risk parameter defined by equation (13) at photoz_X.  photoz_std_X  Second-order moment around a point estimate (photoz_X) derived from full PDF.  photoz_err68_min  16th percentile in the PDF.  photoz_err68_max  84th percentile in the PDF.  photoz_err95_min  2.5th percentile in the PDF.  photoz_err95_max  97.5th percentile in the PDF.  View Large FRANKEN-Z model_llmin: $$-2\ln [\max ({\cal L}_i)]=\min [\chi ^2_n(i)-n(i)]$$, where n(i) = 5 is the number of bands used in the fit. model_levidence: $$-2\ln ({\rm evidence})=-2\ln (\sum _i {\cal L}_i)$$, where $${\cal L}_i = \exp \lbrace -0.5[\chi _n^2(i)-n(i)]\rbrace$$ and the sum over i is taken over all unique neighbors. model_ntype: Number of unique neighbors used in the fit grouped by redshift type (spec, g/prism, and many-band photo-z). model_ptype: Fraction of normalized likelihood contributed by each redshift type. model_nsurvey: As above, but grouped by parent survey (SDSS, etc.). model_psurvey: As above, but contributed by each parent survey. Mizuki reduced_chisq, $$\chi ^2_\nu$$: Reduced chi-squares of the best-fit model. It is recommended to remove objects having $$\chi ^2_\nu >5$$ for scientific use. stellar_mass: Median stellar mass derived from P(M*), which is the stellar mass PDF marginalized over all the other parameters. The 68% confidence intervals are also available. All the uncertainties on physical parameters include uncertainties from photo-z’s. sfr: Median star formation rate with 68% intervals. tauv, τV: Median dust attenuation in the V band with 68% intervals. Note that AV = 1.09τV. prob_x: x is either gal, qso, or star, which denote the relative probability that an object is a galaxy, a QSO, or a star. rest-frame magnitudes: Rest-frame magnitudes in the GALEX, SDSS, HSC, and WFCAM filters. Only the magnitudes from the best-fit template at the median redshifts are computed, and no uncertainties are currently available. MLZ flux_binary_flag: Binary flag to show how many CModel fluxes at different filters are available,   \begin{equation} {\rm f} = \sum _{i=0}^4 \left\lbrace 2^{9-i}{\it PF}_i + 2^{4-i} {\mathit NF}_i \right\rbrace , \end{equation} (21)where PFi = 1 if fluxi > σfluxi, and NFi = 1 if |fluxi| > σfluxi, and 0 otherwise. Index i denotes filters with 0 being g band and 4 being y band. If the object is measured well in all five bands, the flag has value 1023. All of the catalog products such as photo-z point estimates are available in the database. The full PDFs are stored in the FITS format, and are available from the photo-z page of the PDR1 site. 8 Discussion and summary We have presented the photo-z’s computed with several independent codes using the data from HSC-SSP. We have constructed the training sample by combining spec-z, grism-z, and high-accuracy photo-z, and applied a weight to each object to reproduce the color/magnitude distribution of galaxies in the Wide layer. The codes are trained, validated, and tested using this training sample. We also use the COSMOS Wide-depth stacks, in which the photometry is quasi-independent of the training sample, in order to evaluate the seeing and depth dependence of our photo-z performance. We have compared the performance between the codes in section 5. There are trends common to all the codes, such as: (1) our photo-z’s are most accurate at 0.2 ≲ z ≲ 1.5 where we can straddle the 4000 Å break with our filter set, and (2) accuracy is nearly constant at i ≲ 23 and becomes worse at fainter magnitudes. We use a few different algorithms in our machine-learning codes (i.e., neural network, nearest neighbor, self-organizing map), but all the machine-learning codes perform better than the classical template-fitting code (Mizuki). Although this may not be a firm, general conclusion because we have only one template-fitting code (and it was trained against an old version of the training sample with problematic weights), this may have implications for our future photo-z strategy. It is not a surprising result that machine learning outperforms the classical template fitting. There are multiple reasons for this. One would be that template-fitting codes suffer directly from systematic effects in the photometry such as less accurate CModel photometry at bright magnitudes (see Aihara et al. 2018b), while machine-learning codes make the empirical mapping between the photometry and redshift including such systematic effects. Machine-learning codes are thus less prone to systematic effects. However, in order to train a machine-learning code, we need an unbiased training sample. This is a fundamentally difficult problem because photometry always goes deeper than spectroscopy (at least with the current detector technology), and there is no complete spectroscopic sample down to faint enough (e.g., i = 25) magnitudes. There are ongoing efforts to mitigate the problem, which will be useful for weak-lensing science, in which only relatively bright galaxies are used. However, in the UltraDeep layers of HSC-SSP, for instance, we reach deeper than i = 27, where we have few spectroscopic redshifts. While further spectroscopic efforts are definitely needed, another way to mitigate the problem would be to combine the template fitting and machine learning. We can first use the template-fitting technique with photometry in many filters. If our understanding of galaxy SEDs is reasonable, we can assume that these many-band photo-z’s are relatively accurate even beyond the depth of the spectroscopic limit. We can then train machine-learning codes against these many-band photo-z’s using much fewer filters to compute photo-z’s over a wide area. In fact, this is exactly what we did in our photo-z training; we trained our five-band photo-z’s against the COSMOS many-band photo-z catalog (Laigle et al. 2016). However, there are problems in the current dataset. First, the current optical data in COSMOS used in the photo-z calculation is not very deep, roughly 30–60 min integration, and it is not quite deep enough to train our codes for the Wide survey with 20 min integration. Fortunately, the UltraDeep COSMOS data from HSC-SSP is much deeper, which will solve this problem. Another problem is that COSMOS is currently the only wide enough field observed in many filters, and high-accuracy photo-z’s are available. As discussed in subsection 6.1, there are significant large-scale structures even in COSMOS with multiple redshift peaks. We have reweighted the training sample to reproduce the multi-color distribution of galaxies in the Wide layer, which largely reduces the effects of large-scale structures in COSMOS. However, it will still be very useful to have multiple COSMOS-like fields to suppress any field-specific systematics. UDS may be the next COSMOS field, given its deep optical to IR data over a wide area, although intensive spectroscopic efforts are unfortunately missing in the field. There are also very narrow spikes in the N(z) distribution of COSMOS, which are likely to have been introduced by attractor solutions in the photo-z code and are not accounted for by the reweighting. We need to run multiple template-fitting codes, not just one, to suppress such systematics. We should also resort to clustering techniques to circumvent the problem. There are ongoing efforts on clustering-based N(z) estimations in HSC, and we hope to report on that in a future paper. The technique does not suffer from any problems with photometry as it only requires positional information. A dense spectroscopic sample over the entire redshift range is needed, but SDSS already offers this, at least for tests of N(z) reconstruction in the Wide layer. We could also apply the technique to validate the many-band photo-z’s at very faint magnitudes, where no spectroscopic data is available, to check how reliable many-band photo-z’s are beyond the reach of spectroscopic sensitivities. It is an open question how to handle the evolution of the galaxy bias, but clustering-based redshift inference is certainly a promising way forward. We have focused on redshifts in this paper, but there is other information we would need for science, such as stellar mass and star formation rates of galaxies. A template-fitting code delivers such information, but we could also train machine-learning codes to compute these physical properties. The training sample will again come from COSMOS-like fields, and we probably need to run multiple codes with templates from multiple stellar population synthesis codes in order to have a sense of the systematics in the physical properties. This will also be our next step. Aside from the problem of the training sample, there is another question of whether we should “synthesize” photo-z estimates from all the codes into one, master photo-z. We probably should do so since the photo-z synthesis hopefully reduces uncertainties in each photo-z estimate under the assumption that not all the codes make the same mistake. It is also good for users to have just one photo-z for each object. Our preliminary analysis performed in an earlier photo-z production run suggests that, when there is a code that performs significantly better than the others, that code tends to dominate the master photo-z. However, in this release, most of the codes perform equally well and it is probably worth testing photo-z synthesis again. This is another future task of the HSC photo-z group. Finally, we remind readers once again that the photo-z products discussed in this paper are publicly available. The photo-z point estimates, and confidence and risk parameters, as well as other ancillary information, are all stored in the database. A full P(z) for each object is available in the FITS format, and can be downloaded from the photo-z page on the data release site. Some of our codes suffered from sub-optimal weights used in training, and also from over-training. We hope to mitigate these issues and release improved versions of our photo-z products in a future incremental release of HSC-SSP. Acknowledgements The Hyper Suprime-Cam (HSC) collaboration includes the astronomical communities of Japan and Taiwan, and Princeton University. The HSC instrumentation and software were developed by the National Astronomical Observatory of Japan (NAOJ), the Kavli Institute for the Physics and Mathematics of the Universe (Kavli IPMU), the University of Tokyo, the High Energy Accelerator Research Organization (KEK), the Academia Sinica Institute for Astronomy and Astrophysics in Taiwan (ASIAA), and Princeton University. Funding was contributed by the FIRST program from the Japanese Cabinet Office, the Ministry of Education, Culture, Sports, Science and Technology (MEXT), the Japan Society for the Promotion of Science (JSPS), the Japan Science and Technology Agency (JST), the Toray Science Foundation, NAOJ, Kavli IPMU, KEK, ASIAA, and Princeton University. This paper makes use of software developed for the Large Synoptic Survey Telescope. We thank the LSST Project for making their code available as free software at ⟨http://dm.lsst.org⟩. The Pan-STARRS1 surveys (PS1) have been made possible through the contributions of the Institute for Astronomy, the University of Hawaii, the Pan-STARRS Project Office, the Max-Planck Society and its participating institutes, the Max Planck Institute for Astronomy, Heidelberg and the Max Planck Institute for Extraterrestrial Physics, Garching, The Johns Hopkins University, Durham University, the University of Edinburgh, Queen’s University Belfast, the Harvard-Smithsonian Center for Astrophysics, the Las Cumbres Observatory Global Telescope Network Incorporated, the National Central University of Taiwan, the Space Telescope Science Institute, the National Aeronautics and Space Administration under Grant No. NNX08AR22G issued through the Planetary Science Division of the NASA Science Mission Directorate, the National Science Foundation under Grant No. AST-1238877, the University of Maryland, Eotvos Lorand University (ELTE), and the Los Alamos National Laboratory. This paper is based on data collected at the Subaru Telescope and retrieved from the HSC data archive system, which is operated by the Subaru Telescope and Astronomy Data Center at the National Astronomical Observatory of Japan. We thank the COSMOS team for making their private spectroscopic redshift catalog available for our calibrations. MT acknowledges the support of JSPS KAKENHI Grant Number JP15K17617. AN is supported in part by MEXT KAKENHI Grant Number 16H01096. JSS is supported by the National Science Foundation Graduate Research Fellowship under Grant No. 2016222625. Any opinion, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. HM was supported by the U.S. DOE under Contract DE-AC02-05CH11231, and by the NSF under grants PHY-1316783 and PHY-1638509. HM was also supported by JSPS Grant-in-Aid for Scientific Research (C) No. 17K05409, Scientific Research on Innovative Areas (No. 15H05887), and by WPI, MEXT, Japan. We thank Jun Nakano for discussions on the training of machine-learning codes. We also thank the anonymous referee for a very useful report, which helped improve the paper. Appendix 1. Previous internal photo-z releases As summarized in Aihara et al. (2018b), we have made five internal data releases. For each release, the HSC photo-z working group computed photo-z’s using several independent codes and released the photo-z products to the collaboration. As these internal photo-z products are used in our science papers, we briefly summarize them here. This paper is based on the photo-z products in the S16A internal data release (i.e., the latest release at the time of writing). In the current release, we have used six codes. However, we started with four codes (DEmP, MLZ, Mizuki, and LePhare) in the first data release (S14A0). FRANKEN-Z was included in S15B, and EPHOR in S16A. For MLZ, random-forest was used to compute photo-z’s until S15B and it changed to SOM in S16A. In the early runs, we used a template-fitting code, LePhare, but it was later replaced with NNPZ, which performs better. There have been incremental updates in all the codes in each release, which helped to steadily improve our photo-z performance over the years. However, the performance in the earlier runs is not drastically different from that presented in this paper. Thus, the accuracy quoted in this paper can be used as a rough reference to our previous releases. Once again, the photo-z’s for PDR1 are based on the S16A internal release. Our calibration strategy in earlier releases was similar to the one presented in this paper, but we almost exclusively relied on the many-band photo-z’s from COSMOS. We cross-matched the HSC objects with the COSMOS photo-z catalog by position and split it into two: training+validation and test. Each photo-z runner used the first sample to train and validate the code and applied the trained code to the second sample to test the performance. While this approach worked well for faint objects, bright nearby objects were under-represented in COSMOS and we discovered problems with low-z objects in the Wide area. This led to the combined sample of a bright spec-z sample and a faint photo-z sample used in the training in this paper. Also, the best point estimator and the risk parameter were first introduced in S16A and in this paper, and were not used in our previous releases. Most papers based on our previous photo-z products use zmedian and C(zmedian) instead. Appendix 2. Biases and scatter in the physical parameter estimates by Mizuki Mizuki infers physical properties of galaxies such as stellar mass and star formation rates (SFRs) self-consistently in addition to redshifts. This section evaluates how accurate the physical parameter estimates are. For this goal, we use data from the Newfirm Medium Band Survey (NMBS: Whitaker et al. 2011). We focus here on the AEGIS field, and use the stellar mass and SFR estimates by Whitaker et al. (2011) based on the NMBS and multi-wavelength data available in AEGIS. Figure 15 compares stellar mass and SFR from Mizuki against NMBS. As shown in the main body of the paper, our photo-z’s are not very accurate at z ≳ 1.5, where we lose the 4000 Å break, and we focus on galaxies at z < 1.5 here. Note that the redshift is not fixed to those from NMBS, but left as a free parameter. Overall, our stellar mass and SFR agree well with those from NMBS over the entire plotted range with a scatter of about 0.25 dex, including photo-z errors. However, there is a systematic bias: stellar mass is overestimated by 0.2 dex, and SFR underestimated by 0.1 dex. These biases in the physical properties are a function of redshift, as shown in the top panels. The biases are likely to be due to a combination of the template error functions and physical priors applied (Tanaka 2015). Work is in progress to reduce the systematic biases, but we note that a level of 0.3 dex biases is relatively common in this field; van Dokkum et al. (2014) found a relatively large stellar mass offset of 0.2 ∼ −0.3 dex between the 3D-HST and UltraVISTA catalogs even though both catalogs have deep photometry in many filters. Part of the bias we observe here might come from systematics in the data (either in HSC or NMBS). Fig. 15. View largeDownload slide Stellar mass (left) and SFR (right) from Mizuki plotted against those from NMBS. The top panel in each plot shows the ratio between Mizuki and NMBS as a function of redshift. The dashed lines show perfect correspondence, and the dotted lines show the mean bias. (Color online) Fig. 15. View largeDownload slide Stellar mass (left) and SFR (right) from Mizuki plotted against those from NMBS. The top panel in each plot shows the ratio between Mizuki and NMBS as a function of redshift. The dashed lines show perfect correspondence, and the dotted lines show the mean bias. (Color online) Footnotes 1 ⟨https://hsc-release.mtk.nao.ac.jp/doc/index.php/photometric-redshifts/⟩. 2 It can be found at ⟨https://github.com/joshspeagle/frankenz⟩. 3 In the catalog database at the data release site, this parameter is named photoz_risk. 4 foutlier is larger for best than for median, but this is due to reduced scatter (σ). Recall that foutlier is defined as 2 σ outliers. References Abazajian K. et al.   2003, AJ , 126, 2081 CrossRef Search ADS   Aihara H. et al.   2018a, PASJ ,  70, S4 Aihara H. et al.   2018b, PASJ ,  70, S8 Alam S. et al.   2015, ApJS , 219, 12 CrossRef Search ADS   Arnouts S., Cristiani S., Moscardini L., Matarrese S., Lucchin F., Fontana A., Giallongo E. 1999, MNRAS , 310, 540 CrossRef Search ADS   Beers T. C., Flynn K., Gebhardt K. 1990, AJ , 100, 32 CrossRef Search ADS   Benítez N. 2000, ApJ , 536, 571 CrossRef Search ADS   Bolzonella M., Miralles J. -M., Pelló R. 2000, A&A , 363, 476 Bonnett C. et al.   2016, Phys. Rev. D , 94, 042005 CrossRef Search ADS   Bordoloi R., Lilly S. J., Amara A. 2010, MNRAS , 406, 881 Bosch J. et al.   2018, PASJ ,  70, S5 Bradshaw E. J. et al.   2013, MNRAS , 433, 194 CrossRef Search ADS   Brammer G. B. et al.   2012, ApJS , 200, 13 CrossRef Search ADS   Brammer G. B., van Dokkum P. G., Coppi P. 2008 ApJ , 686, 1503 CrossRef Search ADS   Breiman L. 2001, Machine Learning , 45, 1 Brescia M. et al.   2016, PASP , 126, 783 Bruzual G., Charlot S. 2003, MNRAS , 344, 1000 CrossRef Search ADS   Calzetti D., Armus L., Bohlin R. C., Kinney A. L., Koornneef J., Storchi-Bergmann T. 2000, ApJ , 533, 682 CrossRef Search ADS   Carliles S., Budavári T., Heinis S., Priebe C., Szalay A. S. 2010, ApJ , 712, 511 CrossRef Search ADS   Carrasco Kind M., Brunner R. J. 2013, MNRAS , 432, 1483 CrossRef Search ADS   Chabrier G. 2003, PASP , 115, 763 CrossRef Search ADS   Coil A. L., Weiner B. J., Holz D. E., Cooper M. C., Yan R., Aird J. 2011, ApJ , 743, 46 CrossRef Search ADS   Collister A., Lahav O. 2004, PASP , 116, 345 CrossRef Search ADS   Cool R. J. et al.   2013, ApJ , 767, 118 CrossRef Search ADS   Cooper M. C. et al.   2011, ApJS , 193, 14 CrossRef Search ADS   Cooper M. C. et al.   2012, MNRAS , 419, 3018 CrossRef Search ADS   Coupon J. et al.   2009, A&A , 500, 981 CrossRef Search ADS   Cunha C. E., Lima M., Oyaizu H., Frieman J., Lin H. 2009, MNRAS , 396, 2379 CrossRef Search ADS   Dahlen T. et al.   2013, ApJ , 775, 93 CrossRef Search ADS   Davis M. et al.   2003, SPIE Proc. , 4834, 161 de Jong J. T. A. et al.   2013, Messenger , 154, 44 Drinkwater M. J. et al.   2010, MNRAS , 401, 1429 CrossRef Search ADS   Feldmann R. et al.   2006, MNRAS , 372, 565 CrossRef Search ADS   Flaugher B. 2005, Int. J. Modern Phys. A , 20, 3121 CrossRef Search ADS   Garilli B. et al.   2014, A&A , 562, 23 CrossRef Search ADS   Heymans C. et al.   2013, MNRAS , 432, 2433 CrossRef Search ADS   Hildebrandt H. et al.   2010, A&A , 523, 31 CrossRef Search ADS   Hildebrandt H. et al.   2012, MNRAS , 421, 2355 CrossRef Search ADS   Hildebrandt H. et al.   2017, MNRAS , 465, 1454 CrossRef Search ADS   Hildebrandt H., Wolf C., Benítez N. 2008, A&A , 480, 703 CrossRef Search ADS   Hsieh B. C., Yee H. K. C. 2014, ApJ , 792, 102 CrossRef Search ADS   Hsieh B. C., Yee H. K. C., Lin H., Gladders M. D. 2005, ApJS , 158, 161 CrossRef Search ADS   Huang S. et al.   2018, PASJ ,  70, S6 Ilbert O. et al.   2006, A&A , 457, 841 CrossRef Search ADS   Inoue A. K. 2011, MNRAS , 415, 2920 CrossRef Search ADS   Ivezic Z. et al.   2008, arXiv:0805.2366 Kilbinger M. et al.   2013, MNRAS , 430, 2200 CrossRef Search ADS   Kingma D. P., Ba J. 2014, arXiv:1412.6980 Kotulla R., Fritze U., Anders P. 2009, Ap&SS , 324, 347 CrossRef Search ADS   Laigle C. et al.   2016, ApJS , 224, 24 CrossRef Search ADS   Laureijs R. et al.   2011, arXiv:1110.3193 Le Fèvre O. et al.   2004, A&A , 428, 1043 CrossRef Search ADS   Le Fèvre O. et al.   2005, A&A , 439, 845 CrossRef Search ADS   Le Fèvre O. et al.   2013, A&A , 559, A14 CrossRef Search ADS   Leistedt B., Hogg D. W. 2017, ApJ , 838, 5 CrossRef Search ADS   Lilly S. J. et al.   2009, ApJS , 184, 218 CrossRef Search ADS   Lima M., Cunha C. E., Oyaizu H., Frieman J., Lin H., Sheldon E. S. 2008, MNRAS , 390, 118 CrossRef Search ADS   Liske J. et al.   2015, MNRAS , 452, 2087 CrossRef Search ADS   McLure R. J. et al.   2013, MNRAS , 428, 1088 CrossRef Search ADS   Masters A. et al.   2015, ApJ , 813, 53 CrossRef Search ADS   Miyazaki S. et al.   2012, Proc. SPIE , 8446, 84460Z Momcheva I. G. et al.   2016, ApJS , 225, 27 CrossRef Search ADS   Newman J. A. et al.   2013, ApJS , 208, 5 CrossRef Search ADS   Polletta M. et al.   2007, ApJ , 663, 81 CrossRef Search ADS   Polsterer K. L., D’Isanto A., Gieseke F. 2016, arXiv:1608.08016 Salvato M. et al.   2009, ApJ , 690, 1250 CrossRef Search ADS   Silverman J. D. et al.   2015, ApJS , 220, 12 CrossRef Search ADS   Singal J., Shmakova M., Gerke B., Griffith R. L., Lotz J. 2011, PASP , 123, 615 CrossRef Search ADS   Skelton R. E. et al.   2014, ApJS , 214, 24 CrossRef Search ADS   Tagliaferri R., Longo G., Andreon S., Capozziello S., Donalek C., Giordano G. 2003, in Neural Nets , ed. Apolloni B. et al.   ( Berlin: Springer), 226 Google Scholar CrossRef Search ADS   Tanaka M. 2015, ApJ , 801, 20 CrossRef Search ADS   Tasca L. A. M. et al.   2017, A&A , 600, A110 CrossRef Search ADS   van Dokkum P. G. et al.   2014, ApJ , 791, 45 CrossRef Search ADS   Walcher J., Groves B., Budavári T., Dale D. 2011, Ap&SS , 331, 1 CrossRef Search ADS   Westera P., Lejeune T., Buser R., Cuisinier F., Bruzual G. 2002, A&A , 381, 524 CrossRef Search ADS   Whitaker K. E. et al.   2011, ApJ , 735, 86 CrossRef Search ADS   Wolf C. 2009, MNRAS , 397, 520 CrossRef Search ADS   © The Author 2017. Published by Oxford University Press on behalf of the Astronomical Society of Japan. All rights reserved. For Permissions, please email: journals.permissions@oup.com http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Publications of the Astronomical Society of Japan Oxford University Press

Loading next page...
 
/lp/ou_press/photometric-redshifts-for-hyper-suprime-cam-subaru-strategic-program-X08RTBWN1R
Publisher
Oxford University Press
Copyright
© The Author 2017. Published by Oxford University Press on behalf of the Astronomical Society of Japan. All rights reserved. For Permissions, please email: journals.permissions@oup.com
ISSN
0004-6264
eISSN
2053-051X
D.O.I.
10.1093/pasj/psx077
Publisher site
See Article on Publisher Site

Abstract

Abstract Photometric redshifts are a key component of many science objectives in the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP). In this paper, we describe and compare the codes used to compute photometric redshifts for HSC-SSP, how we calibrate them, and the typical accuracy we achieve with the HSC five-band photometry (grizy). We introduce a new point estimator based on an improved loss function and demonstrate that it works better than other commonly used estimators. We find that our photo-z’s are most accurate at 0.2 ≲ zphot ≲ 1.5, where we can straddle the 4000 Å break. We achieve σ[Δzphot/(1 + zphot)] ∼ 0.05 and an outlier rate of about 15% for galaxies down to i = 25 within this redshift range. If we limit ourselves to a brighter sample of i < 24, we achieve σ ∼ 0.04 and ∼8% outliers. Our photo-z's should thus enable many science cases for HSC-SSP. We also characterize the accuracy of our redshift probability distribution function (PDF) and discover that some codes over-/underestimate the redshift uncertainties, which has implications for N(z) reconstruction. Our photo-z products for the entire area in Public Data Release 1 are publicly available, and both our catalog products (such as point estimates) and full PDFs can be retrieved from the data release site, ⟨https://hsc-release.mtk.nao.ac.jp/⟩. 1 Introduction In the era of wide and deep imaging surveys, the photometric redshift technique (hereafter photo-z, see Hildebrandt et al. 2010 and references therein) has become compulsory to uncover the large-scale distance and time information of millions (soon billions) of galaxies. While photo-z algorithms and the photometry measurements have improved significantly over the past two decades (Hildebrandt et al. 2008, 2012; Coupon et al. 2009; Dahlen et al. 2013; Bonnett et al. 2016), the challenge of acquiring photo-z estimates accurate enough to meet the requirements of cosmology and galaxy evolution studies continues to motivate the active development of photometry extraction and photo-z algorithms even today. It is now clear that both template-fitting and machine-learning methods are complementary and necessary to compute meaningful photo-z’s. Template-fitting methods (Arnouts et al. 1999; Bolzonella et al. 2000; Feldmann et al. 2006; Brammer et al. 2008; Kotulla et al. 2009) use known galaxy spectral energy distributions (SEDs) and priors (Benitez 2000; Ilbert et al. 2006; Tanaka 2015) to match the observed colors with predicted ones. Such an approach currently represents the only way to provide photo-z estimates in regions of color/magnitude space where no reference redshifts are available (but see also Leistedt & Hogg 2017). Machine-learning methods (Tagliaferri et al. 2003; Collister & Lahav 2004; Lima et al. 2008; Wolf et al. 2009; Carliles et al. 2010; Singal et al. 2011; Brescia et al. 2016) are complementary as they provide efficient photo-z estimates, in terms of speed and precision, but require a training sample that is a fair representation of the galaxy sample of interest, which is often difficult to construct due to missing regions in the multi-color space. Precise photo-z’s are needed to enable the selection of sharp, non-overlapping redshift bins to “slice” the Universe. For example, cosmic shear studies (Kilbinger et al. 2013; Hildebrandt et al. 2017) suffer from galaxies in adjacent redshift bins that dilute the cosmological signal and increase the importance of systematic biases such as the galaxy intrinsic alignments (Heymans et al. 2013). For galaxy evolution studies, it is often important to infer physical properties of galaxies such as stellar mass in addition to redshifts. It is thus crucial to minimize catastrophic photo-z errors that lead to erroneous physical parameters. The accurate characterization of the true underlying redshift distribution of a galaxy sample remains a major challenge in today’s experiments. With samples composed of hundreds of millions of galaxies, systematic biases now largely dominate over statistical errors, and gathering a complete and numerous calibration sample has become increasingly pressing in the context of current and planned large-scale imaging surveys. Recently, significant progress has been made in building fainter spectroscopic redshift (hereafter spec-z) samples, e.g., DEEP2 (Davis et al. 2003; Cooper et al. 2011, 2012; Newman et al. 2013), VVDS (Le Fèvre et al. 2004, 2005, 2013), VUDS (Tasca et al. 2017), and 3D-HST (Skelton et al. 2014; Momcheva et al. 2016). These are complemented by larger but shallower surveys such as VIPERS (Garilli et al. 2014), SDSS (Alam et al. 2015), Wiggle-Z (Drinkwater et al. 2010) and GAMA (Liske et al. 2015). More complete, but with lower redshift resolution, samples are also available from PRIMUS (Coil et al. 2011; Cool et al. 2013), along with many-band photo-z’s from COSMOS (Laigle et al. 2016). In parallel, the community has developed powerful new tools to identify deficiencies in existing spec-z samples (see, e.g., Masters et al. 2015) in order to help focus resources on targeting specific galaxy populations with adequate instruments. Still, additional effort is required to (1)