Judging on thin ice: the effects of group membership on evaluation

Judging on thin ice: the effects of group membership on evaluation Abstract Employment, legal and social settings often expect the objective evaluation of others regardless of race, sex or other group identification. Such objectivity may be undermined by group membership that is difficult to detect. In this paper we exploit an institutional feature of figure skating, namely that figure skaters possess individual styles and technique, rather than their performance being influenced by club-specific characteristics, which allows us to identify the role of favouritism in a context where it is intended to be excluded. We can identify judges being influenced by group identity, in this case club membership and exhibiting favouritism for club members. We discuss the implications of our findings for rule which would help limit the role of favouritism in decision-making. 1. Introduction Rules often mandate objectivity and fairness from those who evaluate others. Impartiality is imperative for objective evaluation of legal issues, employment and other settings, including sports. When evaluating the performance or productivity of competing individuals that has multiple dimensions and requires quick and complex decisions, attributes different from the actual conduct or performance, such as race, gender, religion, or other group identifications may influence assessments. Some group identities, such as those shared by university alumni, fraternity and sorority members, or civic associations such as Masons or Elks clubs, are not always apparent to non-group members yet may also influence evaluators who share group membership. The lack of visibility of those affiliations frustrates attempts to study situations where one might suspect bias and may allow favouritism in many settings to go unnoticed and unstudied. Corporate settings often use evaluation either by one individual or by a committee to make decisions regarding the hiring and advancement of others. Visible group memberships such as race and gender may be addressed through diversity in the decision-making body. However other and less visible group identities cannot be as easily addressed. Well-known work in social psychology analyses group membership in an experimental setting. This work shows that even seemingly meaningless distinctions among groups can lead individuals to favour those within their group relative to those outside of their group (Tajfel, 1970). Economists have contributed to this area of study with theoretical and experimental analyses of the effect of identity and of in-group membership (Akerlof and Kranton, 2000; Bertrand et al., 2005; Goette et al., 2006; Charness et al, 2007; Chen and Li, 2009; Shayo and Zussman, 2011; Bar and Zussman, 2012). Random assignment into real social groups in Goette et al. (2006) provides stronger group identification than mere laboratory artificial groups and fosters greater cooperation rates within groups. In this paper, we analyse how mandatory yet self-chosen group affiliation impacts behaviour. In the sports economics literature, there is an abundance of work on favouritism due to race, gender and ethnicity. This literature includes work showing that individuals are more likely to favourably evaluate those who share the same race (Price and Wolfers, 2010; Parsons et al., 2011). Biased judging due to nationalism is emphasized in analyses of Olympic sports such as diving (Emerson et al., 2009; Emerson and Arnold, 2011), ski jumping and figure skating (Zitzewitz, 2006) and gymnastics (Leskovsek et al., 2012). Olympic figure skating marks using an older scoring system were also found to be subject to national bias (Whissell et al., 1993). Recent studies analyse judging anomalies in the sport of figure skating with a focus on Olympic competitions in which the judges’ marks have been anonymous since 2005 (Looney, 2012; Zitzewitz, 2014).1 A recent study found that umpires in Major League Baseball tended to expand the strike zone in favour of the home team (Kim and King, 2014). This is especially interesting as the umpires are in an environment in which every ball and strike call can be evaluated by impartial technology. In this study, we have identified a potential source of biased judging for skating competition which is the institution of club membership in US Figure Skating. Membership of both skaters and judges in a local skating club is required. One advantage of this setting and the corresponding data is that in contrast to international anonymity, data from US figure skating competitions allows us to identify the club membership of both the skater and skating judges, along with the details of the skating performance and evaluation. The structure of figure skating in the USA provides us with the opportunity to empirically separate characteristics of performance from other characteristics that might result in biased judging. In the USA officials evaluate skating performances according to the International Judging System (IJS), which is a set of objective criteria that was developed by the International Skating Union (ISU). Because judges are randomly assigned to competitions and events in the qualifying competitions up to and including the National Championships, we are not concerned with selection issues that might bias our results if judges could choose to be on the judging panels for skaters that they favour. One benefit of our data is that there is a panel of judges, rather than just one, each of whom performs their evaluation independently, evaluating each skater’s performance. This allows us to control for the quality of the skate, as judged by peers. We show that skaters receive higher marks from judges with whom they share a club affiliation. We find these results for singles skating, senior and non-senior levels of skating ability, and national, sectional and regional events. We also find support for the hypothesis that judges follow a strategy of showing favouritism in cases where the results matter the most, at the point of discontinuity, when skaters are on the verge of elimination from advancement to higher-level competitions. Additionally, we examine evaluation decisions when the guidelines for awarding marks are less strict. Here, we find a larger bias when judges have greater discretion in awarding marks. One concern regarding our analysis might be that skating styles differ by clubs, and that skating judges favour skaters in their club, because they prefer a particular club’s skating style. However, evidence suggests that skaters from different skating clubs do not differ systematically with respect to their style of skating but that individual characteristics and individual style matter in skating. According to Ellyn Kestnbaum’s Culture on Ice: Figure Skating and Cultural Meaning, figure skaters bring their individual strengths and weaknesses to their performances. A skater’s most outstanding abilities could be those of being more athletic, technical or artistic. Judges reward excellence of any kind when they see it. ‘No mathematical formula can capture the entirety of a skating performance. The best ones are designed not simply to showcase elements. … The best skating programs are meant to move us’ (Adams, 2011, p.75). 2. Institutions: skating clubs, skating disciplines, qualifying structure and judges 2.1 Skating clubs Skaters, officials and coaches must all be members of US Figure Skating through a local club to participate in skating competitions. We rely on the assumption, supported by anecdotal evidence, that clubs are not known for skating styles, but for the amenities they offer to skaters and their families and for their geographic locations. The style of a skater is based on individual strengths and weaknesses in combination with personal choices by the skater as to their music, choreographer and techniques. Coaches, parents and other members of a skater's individual ‘team’ assist in decisions regarding the design of choreography, costumes, make-up and hairstyles, etc. When groups maintain specific characteristics, it is difficult to determine whether favouritism in evaluations is due to bias, or due to placing a higher weight on same-group characteristics. Anecdotal evidence suggests that figure skating club membership does not connote any specific same-group similarities (Kestnbaum, 2003). Though there are some locations in which skaters cluster to train with other high calibre, internationally competitive skaters, skaters usually compete for their home club rather than changing their club affiliation to their training location. 2.2 Disciplines, competitions, events and divisions Competitive figure skating has five disciplines: ladies’ singles, men's singles, synchronized team skating, pairs and ice dancing. Singles skaters and teams have unique club affiliations, allowing us to study potential bias in evaluation. In contrast, pair skaters and members of ice-dance teams may represent different clubs, or are located in different parts of the country. Hence, for the latter disciplines, skaters do not always share a group membership, and this is why we exclude pairs and ice-dancing competitions from our analysis. For each figure skating discipline in this study—ladies’ singles and men's singles—a competition has events for skaters of different skill levels. Singles competitors have one of five skill levels. These skill levels are, in order of technical proficiency, juvenile, intermediate, novice, junior and senior. Higher skill levels require the performance of more difficult elements. Each skill level is subject to different rules and guidelines. A so-called event in ice skating includes in three dimensions: a competition level, a skill level and the skater’s sex. The USA is divided into nine skating regions. Each holds a regional competition, with events that are open to singles skaters with the appropriate skill levels. The top four skaters from each event in these regional competitions advance to one of the three sectional singles competitions.2 And, within each sectional event, the top four skaters or teams in each of the sectional competitions advance to the annual US National Championship. Approximately 16,000 figure skaters compete in regional competitions each year and about 175 figure skaters participate in the US National Championships. However, for ladies’ and men’s singles, in the years included in this study, only novice, junior and senior skaters compete in the US Championships. Therefore, for singles skaters, we consider only these three skill levels. 2.3 Singles skating Novice, junior and senior singles events have two separate segments: a short programme, skated first and a free skating programme, also referred to as the long programme. The short programme has required moves, referred to as elements and are determined by US Figure Skating for the intermediate skill level or by the ISU for the novice through senior skill levels. A random draw determines the skating order in the short programme—that is, who skates first, second and so on. For the free skating programme, a seeded draw determines the skating order. Here, skaters are grouped in the inverse order of their rank after the short programme. No rule determines who advances from National Championships to international championships. A US Figure Skating committee decides who of the top skaters at the National Championship will represent the USA at the next year’s international competitions. This includes the opportunity to compete at the World Championships, which awards cash prizes totaling over $700,000, with $45,000 being awarded to the men’s and ladies’ event winners. 2.4 Skating officials and scoring Judges award marks at competitions. Along with the judges, all competitions include a referee, who keeps time, makes sure other officials are at their pre-assigned seats, who stops a performance if equipment malfunctions, etc. All competitions also include a technical panel, which identifies and assigns a level of difficulty to elements. At regional, sectional and national competitions, judges evaluate the skaters in accordance with the guidelines of the IJS. Subsequent to the figure skating judging scandals in the 1998 and 2002 Olympics, the ISU introduced this system in 2004. Prior to 2004, judges awarded ordinal marks for each skater, and final placements were determined by the majority of ordinals. Using the IJS, the technical elements are identified by a technical panel and communicated to the judges as they are performed. For the evaluation of the technical elements, judges have very specific guidelines for markings, with suggested reductions and increases based on performance quality, and mandatory caps on the highest mark when major errors occur. Judges award the technical element marks, also referred to as Grade of Execution (GOE) scores while skaters perform, and award programme component marks directly after each skater or team’s performance. For the programme component score, each judge assigns a mark to each of the five individual components, which are skating skills, transitions, performance/execution, choreography and interpretation. Judges mark each programme component on a scale from 0.25 to 10.0, in increments of 0.25. Table A1 in the Online Appendix shows the computation of skaters’ overall scores. Further details on how scores are calculated is provided in the notes to Table A1. Three competency levels reflect the qualification of judges. For singles skating, the levels are regional, sectional and national. A national appointment indicates a higher level of skill and experience. Judges of all of three competency levels serve in regional competitions. Regional judges cannot serve at sectional or national events and sectional judges cannot serve at national events. Each skater’s club affiliation is announced at competitions just prior to their performance. So, it is implausible that a judge would be unaware that they share a club affiliation with a skater. Since the beginning of the US Figure Skating Association, about 90 years ago, judges have been required to become members through a membership in a local club. Judges receive no pay for their services. However, the clubs reimburse judges for judging-related expenses, including the costs of travel. To avoid a conflict of interest, rules do not allow judges to earn income by serving as a coach, choreographer, or consultant to skaters. The Judges’ Creed Standard of Conduct, which includes the statement ‘I shall make my judgment to the best of my ability with all humility and then shall keep my own counsel unless questioned officially’ (US Figure Skating Rulebook 2012, Section JR1.01, p. 67), is meant to guide the behaviour of judges.3 During the entire event, skating rules do not permit judges to discuss their evaluations among themselves. 3. Model and hypotheses We assume that judges show bias in their evaluation of a skater if the utility of having a member of their group place higher in a competition exceeds the cost of exercising such a preference. Based on this simple assumption, we generate multiple hypotheses regarding how judges evaluate skaters. Hypothesis 1 Judges will increase their marks when they and the skater belong to the same skating club. Testing for biased evaluation based on a shared group membership is different from testing for bias based on race or gender. For race or gender, favouritism may be due to heritable shared genetics rather than a choice of affiliation. In contrast, clubs are social constructs and individuals choose to belong to a particular club. We hypothesize that a social construct, club affiliation, gives rise to biased decision-making. Biased evaluation might also originate from geographical location. That is, judges may evaluate skaters more favourably when the distance between their residence and that of the skater’s club is less. Biased judging based on geography might also occur based on whether the judge’s residence is located in the athlete’s skating region, as defined by US Figure Skating Our first hypothesis predicts that judges will award higher marks to skaters who share the same club affiliation as the judge. We refer to the higher marking by a judge as affiliation bias, when this judge has the same the same club affiliation as the skater. Hypothesis 2 Affiliation bias in evaluation increases when a skater is on the cusp of elimination from further competitions. The top placed four males and four females from each skill level (novice, junior and senior) advance from the regional qualifying competition to the sectional and from the sectional to the national competition. Fifth placed skaters do not advance to the next level of competition. This non-linearity in the benefits of a skater’s final placement alters the cost-benefit analysis for biased judging. The short programmes are skated a day or two before the free skating programmes and the results are publicly available. The fourth-placed skater after the short programme faces the danger of not advancing. A judge who shares a club affiliation with such a skater ‘on the bubble’ has an increased incentive to show favouritism. We predict that skaters in fourth and fifth place after the short programme at regionals and sectionals competitions will receive higher marks from judges of their own club than skaters more securely at the top of the standings, or at the bottom of the standings after the short programme. Hypothesis 3 Evaluation of technical elements will be more objective than the programme component marks. Judges have much more discretion in marking components of a performance, such as choreography, than in marking the grades of execution of the technical elements. While GOE marks must be reduced for a fall or a major error, there is no corresponding requirement for programme component marks. Additionally, GOE marks have a narrower range and have larger, and thus fewer, increments than the component marks. The fact that the increments are larger for GOE marks serves as a barrier to exercising bias in awarding GOE marks, because an increment increase in GOE marks are more easily detected as being due to bias or error, than component marks that come in smaller increments. For these reasons, we predict to identify less bias in marks for the GOE of technical elements than for the marks associated with the five programme components. 4. Data sources and empirical model The institutions of figure skating allow us to examine the group biases based on a quasi-random assignment. US Figure Skating randomly assigns judges each year to each of regional and sectional competitions, as well as to the National Championships. The constraints to the random assignment are eligibility and availability of judges. There are multiple competitions with similar timeframes, implying that manipulating one’s availability is no guarantee of an assignment to a particular competition. Additionally, a judge is assigned to qualifying competitions for the season (October to January) in the prior summer, but is not assigned to specific events until just prior to the beginning of the competition. Thus, we expect no selection issues, as would be the case if judges could choose events in which affiliated skaters are participating. In addition, all judges must be members of US Figure Skating, primarily through membership in a local club. Thus, no selection bias comes from some judges choosing to have a club affiliation while others do not. Further, there is no link between the quality of skaters from a given club and the tendency of judges to belong to that club, that is, judges rarely change affiliations for reasons other than relocation. In our data no judge changed his or her club affiliation.4 In some ways, our data mimic those of a randomized experiment, however, a study like ours that uses field data certainly has shortcomings when compared to one using experimental data. In a well-designed experiment, we can attribute observed differences in behaviour solely to group assignment. This is also the goal of our empirical specification. In our design, judges who do not share club affiliation serve as a control group, allowing us to identify bias arising from shared club affiliation. We collected data from the US Figure Skating website (http://www.usfsa.org [accessed 1 March 2015]) and its affiliate Icenetwork (http://web.icenetwork.com/home [accessed 4 October 2014]). We obtained data on the official competition results for all final rounds of qualifying competitions leading to the National Championships in 2012, and for the 2010, 2011 and 2012 National Championships. Our data allow us to link each skater’s marks to the judge who awarded those marks. We identified the club affiliation of the judges by searching skating club websites, searching the Internet for information regarding their biographies and conducting personal interviews. We obtained the club affiliation of skaters and teams from official competition results. We measure distance between the judge’s residence and the club’s location by the distance between their respective ZIP codes. Information regarding judges’ home ZIP codes comes from the Directory of US Figure Skating Officials 2011–12. One approach to identify bias is to analyse simply whether a judge gives higher scores to skaters with which they share a club membership. This method will result in biased estimates if skaters from an individual judge’s club are better skaters. A second approach to test for bias is to examine the difference between the particular marks of a judge for a skater performance with a shared club affiliation, and the untrimmed mean of all marks given by other judges for the same performance. Previous work in this area uses the average of all judges’ scores as a proxy for the unobservable ‘true’ quality of the performance (Emerson and Meredith, 2011) However, when using this approach, there is a potential for endogeneity. Instead we address endogeneity issues by using indicators to control for the quality of the specific performance, the marks for each judge and other dummy variables. To test our hypotheses, our preferred specification is: markijk=β1SameClubijk+β2Locationij+γi+δj+εijk (1) where markijk is the mark given for a specific performance i by judge j for component k. We employ a similar specification to test the GOE marks, wherein markijk is the GOE mark given for perfomance i by judge j for element k. To test whether any potential favouritism arises based on club affiliation, geographical preferences, or both, we include the variable SameClubijk, which equals one if the judge shares a club affiliation with the skater or skating team. Key to our identification strategy is the assumption that skating styles vary by skater, not by club. As we have described, clubs are merely social entities that provide the right to train on a specific rink at a specific time. We are assuming that no club-specific or geography-specific skating styles exist in figure skating. The variable Locationijkt measures the geographic location of the skater and judge. We use two alternative indicators for geographic location. In one specification, we include an indicator for whether or not the judge resides in the same skating region in which the skater's club is located. In the other specification, we include the distance between the judge's residence and the skater's club location. We include performance fixed effect γi to control for a skater’s overall skill and ability and for any ‘home field’ advantage to the skater. In most specifications we also include judge fixed effects, δj, that captures heterogeneity among judges. We estimate various specifications of model (1) for singles free skating programmes and singles short programmes. We analyse both the programme component marks and the GOE marks. In some specifications, we also study biased judging based on the shared gender of skater and judge. In alternative specifications to model (1) we use indicators for competition and skater fixed effects instead of the performance indicator. The alternative includes an indicator for each of the competitions. For example, the regression includes an indicator for the New England Regional competition, for the Southwest Pacific Regional competition and so on. In most specifications we include skater fixed effects which capture a skater’s overall and ability. In the few instances when neither performance nor skater fixed effects are included, we add indicators for whether the skater is a junior or senior with novice being the omitted category. In most specifications we also include an indicator as to whether the programme is a free skate (long) or a short programme. For the programme component scores, each skater receives as many marks as there are judges for each of the five components. This generates multiple observations per skater. However, treating the multiple scores for each skater as independent is not appropriate. The marks awarded to a skater are correlated because all judges evaluate the same performance and share similar views regarding what counts as a high-quality or low-quality performance. Therefore, we cluster the standard errors in two dimensions, by competitor and by judge. For our baseline regression results, we focus on the programme component marks rather than on the GOE marks. Table 1A summarizes the number of programme component marks that judges award for both free skating and short programmes by competition type and by skater and team skill levels for the 2011–12 season. In parentheses, Table 1A also presents the number of programmes skated in each of these categories. Because judging panels are larger in national competitions than in regional or sectional competitions, there are more marks per judging panel in the national competitions. Judges awarded 47,940 programme component marks awarded during the 2011–12 regional, sectional and national competitions, after removing 245 marks for skaters who are independent members of US Figure Skating. Table 1A Cross-tabulation of programme component marks awarded: 2011–12 qualifying season Skill Levels Competition Novice Junior Senior Total (1) (2) (3) (4) (5) National Singles 2,160(48) 2,160(48) 3,510(78) 7,830(174) Sectional Singles 4,315(144) 4,200(140) 3,870(129) 12,385(413) Regional Singles 11,640(388) 8,885(301)** 7,200(240) 27,725(929) Totals 18,115(580) 15,245(489) 14,580(447) 47,940(1,516) Skill Levels Competition Novice Junior Senior Total (1) (2) (3) (4) (5) National Singles 2,160(48) 2,160(48) 3,510(78) 7,830(174) Sectional Singles 4,315(144) 4,200(140) 3,870(129) 12,385(413) Regional Singles 11,640(388) 8,885(301)** 7,200(240) 27,725(929) Totals 18,115(580) 15,245(489) 14,580(447) 47,940(1,516) Source: Authors’ calculations. Note: The unit of analysis is a judge’s programme component mark and data are for the 2011–12 skating season. Each judge awards five marks per programme per skater. Table 1A tabulates the total number of component marks awarded in each level of competition at each level of skating. In parentheses are the total numbers of long and short programmes. Regional and sectional competitions generally use a panel of six judges (**29 programmes judges with a five judge panel) while national competitions use nine judges, resulting in more marks per skater. Table 1A Cross-tabulation of programme component marks awarded: 2011–12 qualifying season Skill Levels Competition Novice Junior Senior Total (1) (2) (3) (4) (5) National Singles 2,160(48) 2,160(48) 3,510(78) 7,830(174) Sectional Singles 4,315(144) 4,200(140) 3,870(129) 12,385(413) Regional Singles 11,640(388) 8,885(301)** 7,200(240) 27,725(929) Totals 18,115(580) 15,245(489) 14,580(447) 47,940(1,516) Skill Levels Competition Novice Junior Senior Total (1) (2) (3) (4) (5) National Singles 2,160(48) 2,160(48) 3,510(78) 7,830(174) Sectional Singles 4,315(144) 4,200(140) 3,870(129) 12,385(413) Regional Singles 11,640(388) 8,885(301)** 7,200(240) 27,725(929) Totals 18,115(580) 15,245(489) 14,580(447) 47,940(1,516) Source: Authors’ calculations. Note: The unit of analysis is a judge’s programme component mark and data are for the 2011–12 skating season. Each judge awards five marks per programme per skater. Table 1A tabulates the total number of component marks awarded in each level of competition at each level of skating. In parentheses are the total numbers of long and short programmes. Regional and sectional competitions generally use a panel of six judges (**29 programmes judges with a five judge panel) while national competitions use nine judges, resulting in more marks per skater. Table 1B presents the cross tabulations for the number of programme component marks awarded for the US National Championship competitions in 2010, 2011 and 2012 for novice, junior and senior levels, broken down by short and free skating programmes. Single skaters perform in both programmes, unless they become injured. Therefore, almost half of the observations for singles are short programme marks while the other half consists of free skating programme marks. Our data for the three years of National Championships include 24,705 observations. Table 1B Cross-tabulation of programme component marks awarded: National Championships 2010, 2011 and 2012 Skill Levels Discipline Programme Length TV Novice Junior Senior Totals Singles Short No 3,240(72) 3,330(74) 5,850(130) 12,420(276) Singles Long No 3,240(72) 3,330(74) 6,570(146) Singles Long Yes 5,715(127) 5,715(127) Totals 6,480(144) 6,660(148) 11,565(257) 24,705(549) Skill Levels Discipline Programme Length TV Novice Junior Senior Totals Singles Short No 3,240(72) 3,330(74) 5,850(130) 12,420(276) Singles Long No 3,240(72) 3,330(74) 6,570(146) Singles Long Yes 5,715(127) 5,715(127) Totals 6,480(144) 6,660(148) 11,565(257) 24,705(549) Source: Authors’ calculations. Note: The unit of analysis is a judge’s programme component mark and data are for the 2010, 2011 and 2012 National Championships. Each judge awards five marks per programme per skater. Table 1B tabulates the total number of component marks awarded in each level of competition at each level of skating. In parentheses are the total numbers of long and short programmes. Table 1B Cross-tabulation of programme component marks awarded: National Championships 2010, 2011 and 2012 Skill Levels Discipline Programme Length TV Novice Junior Senior Totals Singles Short No 3,240(72) 3,330(74) 5,850(130) 12,420(276) Singles Long No 3,240(72) 3,330(74) 6,570(146) Singles Long Yes 5,715(127) 5,715(127) Totals 6,480(144) 6,660(148) 11,565(257) 24,705(549) Skill Levels Discipline Programme Length TV Novice Junior Senior Totals Singles Short No 3,240(72) 3,330(74) 5,850(130) 12,420(276) Singles Long No 3,240(72) 3,330(74) 6,570(146) Singles Long Yes 5,715(127) 5,715(127) Totals 6,480(144) 6,660(148) 11,565(257) 24,705(549) Source: Authors’ calculations. Note: The unit of analysis is a judge’s programme component mark and data are for the 2010, 2011 and 2012 National Championships. Each judge awards five marks per programme per skater. Table 1B tabulates the total number of component marks awarded in each level of competition at each level of skating. In parentheses are the total numbers of long and short programmes. 5. Results 5.1 Group membership and evaluation Starting with cross-tabulations, we first computed the number of observations with the highest mark for each component given to each skater for each programme. We then separated the marks into two groups, those in which the judge and skater share a club affiliation and those in which they do not. Table 2 shows that high marks constitute 22.41% of all marks.5 The highest marks awarded by judges to skaters with whom they share the same club affiliation constitute 29.44% of all high marks. Marks awarded by judges to skaters with whom they do not share the same club affiliation constitute 21.99% of the highest marks. We reject the hypothesis that this difference between two groups is due to chance at the 1% level (binomial p, one-tailed. = 0.00). These results provide initial evidence that judges tend to award the highest marks when they share the same club affiliation with the skater they are evaluating. Table 2 Tabulation of highest programme component mark Number of Marks Shared Club Affiliation All Highest % Highest Marks (1) (2) (3) Yes 2,711 798 29.44 No 45,169 9,936 21.99 Totals 47,880 10,734 22.41 Number of Marks Shared Club Affiliation All Highest % Highest Marks (1) (2) (3) Yes 2,711 798 29.44 No 45,169 9,936 21.99 Totals 47,880 10,734 22.41 Source: Authors’ calculations. Note: The table reports data from the 2011–12 skating season. Column (1) shows the total number of observations and the number of observations where the judge shares the same club affiliation or not. Column (2) considers only the observations that constitute the highest mark per judge per component per programme skated. Column (3) provides percentage of these marks that are given by judges with and without the same club affiliation as the skater. Table 2 Tabulation of highest programme component mark Number of Marks Shared Club Affiliation All Highest % Highest Marks (1) (2) (3) Yes 2,711 798 29.44 No 45,169 9,936 21.99 Totals 47,880 10,734 22.41 Number of Marks Shared Club Affiliation All Highest % Highest Marks (1) (2) (3) Yes 2,711 798 29.44 No 45,169 9,936 21.99 Totals 47,880 10,734 22.41 Source: Authors’ calculations. Note: The table reports data from the 2011–12 skating season. Column (1) shows the total number of observations and the number of observations where the judge shares the same club affiliation or not. Column (2) considers only the observations that constitute the highest mark per judge per component per programme skated. Column (3) provides percentage of these marks that are given by judges with and without the same club affiliation as the skater. Table 3A presents regression results for the 2011–12 season. The dependent variable in this table is the programme component mark. The observations come from all qualifying events: the nine regional singles competitions, three sectional singles competitions, and one national singles championship competition. Table 3B reports the descriptive statistics for the dependent and independent variables for the regression results in Table 3A. For the regressions in Table 3A, the mean component mark is 4.363 and the standard deviation is 1.188.6Table 3B shows that the marks awarded for skating events range from 0.75 to 9.75. The highest mark possible is 10. Table 3A The effect of shared club affiliation on programme component marks: All levels of qualifying competitions (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.136*** 0.123*** 0.164*** 0.160*** 0.255*** 0.235*** 0.141*** 0.152*** (0.032) (0.034) (0.025) (0.026) (0.054) (0.054) (0.034) (0.034) Judge and skater reside in same region 0.041 0.025 0.016 0.021 (0.027) (0.018) (0.039) (0.017) Log distance of skater club and judge’s residence 0.014* 0.006 −0.011 −0.011 (0.008) (0.006) (0.013) (0.013) Programme Length −0.022 −0.022 −0.046** −0.046** (0.023) (0.024) (0.021) (0.021) Same sex judge and skater 0.024 0.024 0.022 0.022 −0.103* −0.102* 0.018 0.018 (0.029) (0.028) (0.026) (0.026) (0.053) (0.054) (0.032) (0.032) Junior skater 0.404*** 0.404*** (0.059) (0.059) Senior skater 0.932*** 0.933*** (0.027) (0.027) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 47,880 47,880 47,880 47,880 47,880 47,880 47,880 47,880 R-squared 0.826 0.826 0.848 0.848 0.406 0.406 0.779 0.779 (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.136*** 0.123*** 0.164*** 0.160*** 0.255*** 0.235*** 0.141*** 0.152*** (0.032) (0.034) (0.025) (0.026) (0.054) (0.054) (0.034) (0.034) Judge and skater reside in same region 0.041 0.025 0.016 0.021 (0.027) (0.018) (0.039) (0.017) Log distance of skater club and judge’s residence 0.014* 0.006 −0.011 −0.011 (0.008) (0.006) (0.013) (0.013) Programme Length −0.022 −0.022 −0.046** −0.046** (0.023) (0.024) (0.021) (0.021) Same sex judge and skater 0.024 0.024 0.022 0.022 −0.103* −0.102* 0.018 0.018 (0.029) (0.028) (0.026) (0.026) (0.053) (0.054) (0.032) (0.032) Junior skater 0.404*** 0.404*** (0.059) (0.059) Senior skater 0.932*** 0.933*** (0.027) (0.027) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 47,880 47,880 47,880 47,880 47,880 47,880 47,880 47,880 R-squared 0.826 0.826 0.848 0.848 0.406 0.406 0.779 0.779 Source: Authors’ calculations. Note: The dependent variable is the programme component mark awarded by each judge to each skater for each component. The data comprises 13 competitions that make up the 2011–12 qualifying stream in singles. The following events are included: CP Regional 2012, EGL Regional 2012, Eastern Sectional 2012, Midwestern Sectional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, Nationals 2012, Pacific Coast Sectional 2012, SA Regional 2012, SW Regional 2012, UGL Regional 2012. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Table 3A The effect of shared club affiliation on programme component marks: All levels of qualifying competitions (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.136*** 0.123*** 0.164*** 0.160*** 0.255*** 0.235*** 0.141*** 0.152*** (0.032) (0.034) (0.025) (0.026) (0.054) (0.054) (0.034) (0.034) Judge and skater reside in same region 0.041 0.025 0.016 0.021 (0.027) (0.018) (0.039) (0.017) Log distance of skater club and judge’s residence 0.014* 0.006 −0.011 −0.011 (0.008) (0.006) (0.013) (0.013) Programme Length −0.022 −0.022 −0.046** −0.046** (0.023) (0.024) (0.021) (0.021) Same sex judge and skater 0.024 0.024 0.022 0.022 −0.103* −0.102* 0.018 0.018 (0.029) (0.028) (0.026) (0.026) (0.053) (0.054) (0.032) (0.032) Junior skater 0.404*** 0.404*** (0.059) (0.059) Senior skater 0.932*** 0.933*** (0.027) (0.027) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 47,880 47,880 47,880 47,880 47,880 47,880 47,880 47,880 R-squared 0.826 0.826 0.848 0.848 0.406 0.406 0.779 0.779 (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.136*** 0.123*** 0.164*** 0.160*** 0.255*** 0.235*** 0.141*** 0.152*** (0.032) (0.034) (0.025) (0.026) (0.054) (0.054) (0.034) (0.034) Judge and skater reside in same region 0.041 0.025 0.016 0.021 (0.027) (0.018) (0.039) (0.017) Log distance of skater club and judge’s residence 0.014* 0.006 −0.011 −0.011 (0.008) (0.006) (0.013) (0.013) Programme Length −0.022 −0.022 −0.046** −0.046** (0.023) (0.024) (0.021) (0.021) Same sex judge and skater 0.024 0.024 0.022 0.022 −0.103* −0.102* 0.018 0.018 (0.029) (0.028) (0.026) (0.026) (0.053) (0.054) (0.032) (0.032) Junior skater 0.404*** 0.404*** (0.059) (0.059) Senior skater 0.932*** 0.933*** (0.027) (0.027) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 47,880 47,880 47,880 47,880 47,880 47,880 47,880 47,880 R-squared 0.826 0.826 0.848 0.848 0.406 0.406 0.779 0.779 Source: Authors’ calculations. Note: The dependent variable is the programme component mark awarded by each judge to each skater for each component. The data comprises 13 competitions that make up the 2011–12 qualifying stream in singles. The following events are included: CP Regional 2012, EGL Regional 2012, Eastern Sectional 2012, Midwestern Sectional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, Nationals 2012, Pacific Coast Sectional 2012, SA Regional 2012, SW Regional 2012, UGL Regional 2012. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Table 3B Summary statistics for Table 3A Variable N Mean Std. Dev. Min Max Component mark assigned by judge 47,880 4.363 1.188 0.75 9.75 Judge and skater share club affiliation 47,880 0.057 0.231 0 1 Judge and skater reside in same region 47,880 0.458 .498 0 1 Log distance between skater club and judge residence 47,880 5.551 1.526 .0001 8.121 Programme Length 47,880 0.498 0.498 0 1 Junior level skater 47,880 0.318 0.460 0 1 Senior level skater 47,880 0.305 0.461 0 1 Same Sex Judge and Skater 47,880 0.645 0.479 0 1 Variable N Mean Std. Dev. Min Max Component mark assigned by judge 47,880 4.363 1.188 0.75 9.75 Judge and skater share club affiliation 47,880 0.057 0.231 0 1 Judge and skater reside in same region 47,880 0.458 .498 0 1 Log distance between skater club and judge residence 47,880 5.551 1.526 .0001 8.121 Programme Length 47,880 0.498 0.498 0 1 Junior level skater 47,880 0.318 0.460 0 1 Senior level skater 47,880 0.305 0.461 0 1 Same Sex Judge and Skater 47,880 0.645 0.479 0 1 Source: Authors’ calculations. Note: The data comprises 13 competitions that make up the 2011–12 qualifying stream in singles. The following events are included: CP Regional 2012, EGL Regional 2012, Eastern Sectional 2012, Midwestern Sectional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, Nationals 2012, Pacific Coast Sectional 2012, SA Regional 2012, SW Regional 2012 and UGL Regional 2012. Table 3B Summary statistics for Table 3A Variable N Mean Std. Dev. Min Max Component mark assigned by judge 47,880 4.363 1.188 0.75 9.75 Judge and skater share club affiliation 47,880 0.057 0.231 0 1 Judge and skater reside in same region 47,880 0.458 .498 0 1 Log distance between skater club and judge residence 47,880 5.551 1.526 .0001 8.121 Programme Length 47,880 0.498 0.498 0 1 Junior level skater 47,880 0.318 0.460 0 1 Senior level skater 47,880 0.305 0.461 0 1 Same Sex Judge and Skater 47,880 0.645 0.479 0 1 Variable N Mean Std. Dev. Min Max Component mark assigned by judge 47,880 4.363 1.188 0.75 9.75 Judge and skater share club affiliation 47,880 0.057 0.231 0 1 Judge and skater reside in same region 47,880 0.458 .498 0 1 Log distance between skater club and judge residence 47,880 5.551 1.526 .0001 8.121 Programme Length 47,880 0.498 0.498 0 1 Junior level skater 47,880 0.318 0.460 0 1 Senior level skater 47,880 0.305 0.461 0 1 Same Sex Judge and Skater 47,880 0.645 0.479 0 1 Source: Authors’ calculations. Note: The data comprises 13 competitions that make up the 2011–12 qualifying stream in singles. The following events are included: CP Regional 2012, EGL Regional 2012, Eastern Sectional 2012, Midwestern Sectional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, Nationals 2012, Pacific Coast Sectional 2012, SA Regional 2012, SW Regional 2012 and UGL Regional 2012. In all eight columns of Table 3A, the point estimates on skaters and judges sharing the same club affiliation are positive and statistically significant. The estimates show that when a judge and a skater share the same club affiliation, the skater’s mark increases between 0.14 and 0.26 points. Thus, our estimates show that the shared club affiliation leads, on average, to more than half of a one-increment increase in the mark awarded. The regressions in columns 1 and 2 include an indicator for performance to control for the quality of each performance. These first two columns differ with respect to our measure for the location variable. Columns 3 and 4 use identical explanatory variables as the first two columns but add fixed effects for each judge. Columns 1 and 3 include a variable for regional affiliation, identifying whether a judge resides in the same region as the skater's club. Columns 2 and 4 include log distance, measured as the natural logarithm of the number of miles between the residence of the judge and the location of the skater’s club. Columns 5 and 6 include indicators for each competition but do not include skater or judge fixed effects and include indicators for junior and senior levels. Columns 7 and 8 include similar explanatory variables as columns 5 and 6 but include skater and judge fixed effects along with competition fixed effects and drop the dummy variables for junior and senior since these variables are perfectly collinear with the skater fixed effects, given that a skater only competes in one level. We also include an indicator for whether a mark is for a free skating programme in the latter four specifications without performance fixed effects. The point estimates show that having the same club affiliation is associated with an increase in that mark of approximately 15% of the mark’s standard deviation. A conservative estimate is that sharing a club affiliation increases each component mark given by a judge by 2.5%. In a review of the results of qualifying competitions used in this study, almost 15% of event placements are within .25 points of each other. At the 2008 National Championships, Johnny Weir and Evan Lysacek tied for the men’s title. Both skaters earned a total of 244.77 points and a non-points tiebreaker was used. A previous study of Olympic diving by Emerson, Seltzer and Lin (2009) found nationalistic bias in diving results from the 2000 Summer Olympic Games. This study identified specific judges and divers who marked strategically higher or lower than an ‘unbiased dive score’ and found that the removal of judging bias might have changed the medal standings. Although our analysis does not include a similar ‘unbiased’ ranking, the small point differences in skating, between winners and runners-up suggests that bias could impact final results. The most recent study of figure skating (Zitzewitz, 2014) relies on data from international competition which lacks the numerical richness and transparency of the US domestic competitions and thus relied on the max-median and inter-quartile spreads in marks to measure nationalistic bias. Using this method, Zitzewitz found that having a compatriot on panel raised the sum of GOE and components scores by .39 points compared to our finding of an increase in just the components of .15 to .4 points (depending on programme length). Although each judge’s mark contributes toward the average mark, as each judge awards five marks to each skater, the cumulative effect is to increase a skater’s final score by the full amount of the estimated effect. Additionally, the components marks are factored into the total score with .8 of the total used for the short programme but with a factor of 1.6 for a free skate programme. Therefore, the increase in mark is amplified for the free skate programme. In Table 3A, columns 5 and 6 we also find that relative to novices, the average junior skater’s mark is 0.4 points higher and a senior skater’s mark is 0.93 points higher than the average score of novice skaters. This is to be expected as the skill level increases as a skater moves from novice to the higher levels. In light of the results in Table 2, one might wonder whether the results on the shared club affiliation variable are driven by the fact that judges who share a club affiliation with a skater tend to award the highest marks to that skater. To study this issue, we discard for each skater the highest mark for each component. If most of the biased marks are driven by the highest marks, then the impact of bias will be mitigated when we drop the high mark for each component for each programme and use the remaining marks to estimate the same regression specifications as we do in Table 3A. Table 4 shows that the point estimate on shared club affiliation remains positive and statistically significant. We dropped the highest marks, resulting in a new mean of the component marks assigned by a judge of 4.233. Here, a shared club affiliation leads to an increase in the judge’s mark by between 0.09-0.2, these point estimates are of slightly smaller than those in Table 3A. However, the sum of these results shows that the findings in Table 3A are not driven by only those judges who share a club affiliation with the skater and award the highest mark.7 Additional specifications, not reported in this work, which include the difference between one judge’s score and the mean of the other scores on the panel for the same performance, referred to as difference-based analysis (Popović, 2000; Emerson et al., 2009) provide the similar results of statistically significant bias in favour of same club skaters. Table 4 The effect of shared club affiliation on programme component marks: all levels of competition—highest marks excluded (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.094*** 0.087*** 0.127*** 0.126*** 0.199*** 0.186*** 0.127*** 0.141*** (0.030) (0.031) (0.020) (0.021) (0.056) (0.057) (0.031) (0.030) Judge and skater reside in same region 0.042** 0.029** 0.012 0.020 (0.021) (0.014) (0.037) (0.017) Log distance between skater and judge’s residence −0.011 −0.005 −0.008 0.005 (0.007) (0.004) (0.013) (0.007) Programme Length −0.022 −0.023 −0.034* −0.033* (0.025) (0.025) (0.020) (0.020) Same sex judge and skater 0.015 0.014 0.021 0.022 −0.107** −0.107** 0.025 0.024 (0.023) (0.023) (0.021) (0.021) (0.053) (0.053) (0.029) (0.029) Junior skater 0.394*** 0.394*** 0.394*** 0.394*** (0.060) (0.060) (0.060) (0.060) Senior skater 0.925*** 0.925*** 0.925*** 0.925*** (0.099) (0.099) (0.099) (0.099) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 37,146 37,146 37,146 37,146 37,146 37,146 37,146 37,146 R-squared 0.873 0.873 0.887 0.887 0.439 0.439 0.816 0.816 (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.094*** 0.087*** 0.127*** 0.126*** 0.199*** 0.186*** 0.127*** 0.141*** (0.030) (0.031) (0.020) (0.021) (0.056) (0.057) (0.031) (0.030) Judge and skater reside in same region 0.042** 0.029** 0.012 0.020 (0.021) (0.014) (0.037) (0.017) Log distance between skater and judge’s residence −0.011 −0.005 −0.008 0.005 (0.007) (0.004) (0.013) (0.007) Programme Length −0.022 −0.023 −0.034* −0.033* (0.025) (0.025) (0.020) (0.020) Same sex judge and skater 0.015 0.014 0.021 0.022 −0.107** −0.107** 0.025 0.024 (0.023) (0.023) (0.021) (0.021) (0.053) (0.053) (0.029) (0.029) Junior skater 0.394*** 0.394*** 0.394*** 0.394*** (0.060) (0.060) (0.060) (0.060) Senior skater 0.925*** 0.925*** 0.925*** 0.925*** (0.099) (0.099) (0.099) (0.099) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 37,146 37,146 37,146 37,146 37,146 37,146 37,146 37,146 R-squared 0.873 0.873 0.887 0.887 0.439 0.439 0.816 0.816 Source: Authors’ calculations. Note: The dependent variable is the programme component mark awarded by each judge to each skater for each component. The data comprises 13 competitions that make up the 2011–12 qualifying stream in singles. The following events are included: CP Regional 2012, EGL Regional 2012, Eastern Sectional 2012, Midwestern Sectional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, Nationals 2012, Pacific Coast Sectional 2012, SA Regional 2012, SW Regional 2012, UGL Regional 2012. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Table 4 The effect of shared club affiliation on programme component marks: all levels of competition—highest marks excluded (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.094*** 0.087*** 0.127*** 0.126*** 0.199*** 0.186*** 0.127*** 0.141*** (0.030) (0.031) (0.020) (0.021) (0.056) (0.057) (0.031) (0.030) Judge and skater reside in same region 0.042** 0.029** 0.012 0.020 (0.021) (0.014) (0.037) (0.017) Log distance between skater and judge’s residence −0.011 −0.005 −0.008 0.005 (0.007) (0.004) (0.013) (0.007) Programme Length −0.022 −0.023 −0.034* −0.033* (0.025) (0.025) (0.020) (0.020) Same sex judge and skater 0.015 0.014 0.021 0.022 −0.107** −0.107** 0.025 0.024 (0.023) (0.023) (0.021) (0.021) (0.053) (0.053) (0.029) (0.029) Junior skater 0.394*** 0.394*** 0.394*** 0.394*** (0.060) (0.060) (0.060) (0.060) Senior skater 0.925*** 0.925*** 0.925*** 0.925*** (0.099) (0.099) (0.099) (0.099) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 37,146 37,146 37,146 37,146 37,146 37,146 37,146 37,146 R-squared 0.873 0.873 0.887 0.887 0.439 0.439 0.816 0.816 (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.094*** 0.087*** 0.127*** 0.126*** 0.199*** 0.186*** 0.127*** 0.141*** (0.030) (0.031) (0.020) (0.021) (0.056) (0.057) (0.031) (0.030) Judge and skater reside in same region 0.042** 0.029** 0.012 0.020 (0.021) (0.014) (0.037) (0.017) Log distance between skater and judge’s residence −0.011 −0.005 −0.008 0.005 (0.007) (0.004) (0.013) (0.007) Programme Length −0.022 −0.023 −0.034* −0.033* (0.025) (0.025) (0.020) (0.020) Same sex judge and skater 0.015 0.014 0.021 0.022 −0.107** −0.107** 0.025 0.024 (0.023) (0.023) (0.021) (0.021) (0.053) (0.053) (0.029) (0.029) Junior skater 0.394*** 0.394*** 0.394*** 0.394*** (0.060) (0.060) (0.060) (0.060) Senior skater 0.925*** 0.925*** 0.925*** 0.925*** (0.099) (0.099) (0.099) (0.099) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 37,146 37,146 37,146 37,146 37,146 37,146 37,146 37,146 R-squared 0.873 0.873 0.887 0.887 0.439 0.439 0.816 0.816 Source: Authors’ calculations. Note: The dependent variable is the programme component mark awarded by each judge to each skater for each component. The data comprises 13 competitions that make up the 2011–12 qualifying stream in singles. The following events are included: CP Regional 2012, EGL Regional 2012, Eastern Sectional 2012, Midwestern Sectional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, Nationals 2012, Pacific Coast Sectional 2012, SA Regional 2012, SW Regional 2012, UGL Regional 2012. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. 5.2 Evaluation at the discontinuity in payoffs After the free skating programme and short programme scores are summed. The top four men and women, at each skill level advance from the regional competition to the sectional and from the sectional competition to the National Championships, while the season is over for those who finish fifth or lower. After the short programme, which is skated first, some skaters are more secure in advancing based on a strong performance while others, who are ranked very low, have a poor outlook. The skaters in fourth and fifth places after the short programme are on the cusp of advancement but are not guaranteed to advance until performance of the free skating programme and the computation of a final placement. For judges, the benefits of biased marking for these skaters on the bubble are higher in this case than the benefits of biased markings for low-ranked skaters. Therefore, we predict greater bias for skaters on the bubble. To test whether the bubble status affects the marking of the skater we include free skating programme observations for those skaters who were ranked fourth or fifth after the short programme. We include three indicators for the interaction between relationship and placement after the short programme. We do not include an indicator for placement position only because that indicator is perfectly collinear with performance and skater fixed effects. The first three columns of Table 58 show results for regional and sectional competitions. Table 5, column 1 shows approximately a .02-point increase in marks for skaters in fourth or fifth place ‘on the bubble’ and who share the same club affiliation with one of the judges. The point estimate is statistically significant at the 5% level. The positive point estimate implies that when a judge shares the same club membership as the skater, and the skater is on the bubble, this judge increases their mark. The remaining columns in Table 5 show that this result is robust to alternative regression specifications. These findings are consistent with the hypothesis that the largest bias occurs in situations where the bias is most likely to be a determining factor as to whether the skater will advance to the next level of competition. The point estimates for skaters in places 1–3 or 6–8 are not statistically significant, and we do not find evidence of bias for these groups of skaters when a judge shares a same club relationship with the skater. However, in part due to relatively large standard errors estimated for the skaters in places 1–3 and 6–8, an F-test does not allow us to reject that the coefficients for skaters placed 1–3 vs 5–6 are statistically different from each other. Nor can we reject the null hypothesis that skaters placed 6–8 vs 4–5 are statistically different from each other. In both tests, the p-value associated with the F-test is 0.11. Table 5 Strategic marking behaviour. Examination of skaters’ free skating scores after placement in short programme Regional and Sectional Competitions National National (1) (2) (3) (4) (5) Judge and skater share club affiliation 0.128*** 0.055 0.056 0.131* 0.132* (0.045) (0.063) (0.063) (0.076) (0.076) Short Programme Placement 1–3 interaction with shared club 0.003 0.120 0.119 0.147 0.149* (0.067) (0.096) (0.097) (0.099) (0.079) Short Programme Placement 4–5 interaction with shared club 0.022** 0.191** 0.191** 0.081 0.079 (0.011) (0.093) (0.093) (0.261) (0.265) Short Programme Placement 6–8 interaction with shared club −0.103 −0.031 −0.033 −0.012 0.040 (0.078) (0.106) (0.107) (0.000) (0.000) Judge and skater reside in same region 0.037 0.019 0.019 0.048 0.048 (0.025) (0.025) (0.024) (0.062) (0.062) Same sex judge and skater 0.003 −0.016 0.139*** 0.138*** (0.035) (0.050) (0.053) (0.052) Performance Fixed Effect? Yes No No Yes No Skater Fixed Effect? No Yes Yes No Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Competition Fixed Effect? No Yes Yes No Yes Observations 19,945 19,945 19,945 3,915 3,915 R-squared 0.734 0.734 0.734 0.877 0.877 Regional and Sectional Competitions National National (1) (2) (3) (4) (5) Judge and skater share club affiliation 0.128*** 0.055 0.056 0.131* 0.132* (0.045) (0.063) (0.063) (0.076) (0.076) Short Programme Placement 1–3 interaction with shared club 0.003 0.120 0.119 0.147 0.149* (0.067) (0.096) (0.097) (0.099) (0.079) Short Programme Placement 4–5 interaction with shared club 0.022** 0.191** 0.191** 0.081 0.079 (0.011) (0.093) (0.093) (0.261) (0.265) Short Programme Placement 6–8 interaction with shared club −0.103 −0.031 −0.033 −0.012 0.040 (0.078) (0.106) (0.107) (0.000) (0.000) Judge and skater reside in same region 0.037 0.019 0.019 0.048 0.048 (0.025) (0.025) (0.024) (0.062) (0.062) Same sex judge and skater 0.003 −0.016 0.139*** 0.138*** (0.035) (0.050) (0.053) (0.052) Performance Fixed Effect? Yes No No Yes No Skater Fixed Effect? No Yes Yes No Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Competition Fixed Effect? No Yes Yes No Yes Observations 19,945 19,945 19,945 3,915 3,915 R-squared 0.734 0.734 0.734 0.877 0.877 Source: Authors’ calculations. Note: Observations include free skating (long) programmes for the nine regional and three sectional competitions in columns 1–3. Dependent variable is the mark a judge assigns for a programme component. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Table 5 Strategic marking behaviour. Examination of skaters’ free skating scores after placement in short programme Regional and Sectional Competitions National National (1) (2) (3) (4) (5) Judge and skater share club affiliation 0.128*** 0.055 0.056 0.131* 0.132* (0.045) (0.063) (0.063) (0.076) (0.076) Short Programme Placement 1–3 interaction with shared club 0.003 0.120 0.119 0.147 0.149* (0.067) (0.096) (0.097) (0.099) (0.079) Short Programme Placement 4–5 interaction with shared club 0.022** 0.191** 0.191** 0.081 0.079 (0.011) (0.093) (0.093) (0.261) (0.265) Short Programme Placement 6–8 interaction with shared club −0.103 −0.031 −0.033 −0.012 0.040 (0.078) (0.106) (0.107) (0.000) (0.000) Judge and skater reside in same region 0.037 0.019 0.019 0.048 0.048 (0.025) (0.025) (0.024) (0.062) (0.062) Same sex judge and skater 0.003 −0.016 0.139*** 0.138*** (0.035) (0.050) (0.053) (0.052) Performance Fixed Effect? Yes No No Yes No Skater Fixed Effect? No Yes Yes No Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Competition Fixed Effect? No Yes Yes No Yes Observations 19,945 19,945 19,945 3,915 3,915 R-squared 0.734 0.734 0.734 0.877 0.877 Regional and Sectional Competitions National National (1) (2) (3) (4) (5) Judge and skater share club affiliation 0.128*** 0.055 0.056 0.131* 0.132* (0.045) (0.063) (0.063) (0.076) (0.076) Short Programme Placement 1–3 interaction with shared club 0.003 0.120 0.119 0.147 0.149* (0.067) (0.096) (0.097) (0.099) (0.079) Short Programme Placement 4–5 interaction with shared club 0.022** 0.191** 0.191** 0.081 0.079 (0.011) (0.093) (0.093) (0.261) (0.265) Short Programme Placement 6–8 interaction with shared club −0.103 −0.031 −0.033 −0.012 0.040 (0.078) (0.106) (0.107) (0.000) (0.000) Judge and skater reside in same region 0.037 0.019 0.019 0.048 0.048 (0.025) (0.025) (0.024) (0.062) (0.062) Same sex judge and skater 0.003 −0.016 0.139*** 0.138*** (0.035) (0.050) (0.053) (0.052) Performance Fixed Effect? Yes No No Yes No Skater Fixed Effect? No Yes Yes No Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Competition Fixed Effect? No Yes Yes No Yes Observations 19,945 19,945 19,945 3,915 3,915 R-squared 0.734 0.734 0.734 0.877 0.877 Source: Authors’ calculations. Note: Observations include free skating (long) programmes for the nine regional and three sectional competitions in columns 1–3. Dependent variable is the mark a judge assigns for a programme component. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Columns 4 and 5 of Table 5 shows results for national competitions, where we do not predict an effect on the interaction variable, since no skater automatically advances to international competitions. Consistent with this prediction, for the National Championship, we do not find a statistically significant effect on the interaction variable between the fourth or fifth placement and shared club variables. We find an increase in marks awarded when the skater and judge share a club affiliation and when skater and judge have a shared sex. However, since skaters only compete against skaters of the same sex, this would not be significant in determining final placements. 5.3 Effect of level of competition on affiliation bias To examine whether biased judging is occurring only at some types of competitions, in Table 6 we estimate separate regressions for each competition level for the 2011–12 singles event with the programme component mark as the dependent variable. All specifications include either performance and judge fixed effects that control for heterogeneity of judging and skating ability or competition, skater and judge fixed effects as an alternative method of controls. Further, as in the previous regressions, we cluster standard errors by both judge and competitor. Table 6 The effect of shared club affiliation on programme component marks: all levels of qualifying competitions, singles only, 2011–12 Regional Sectional National (same as table 7) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.190*** 0.172*** 0.172*** 0.100* 0.104* 0.101* 0.127** 0.207*** 0.110* (0.028) (0.039) (0.039) (0.054) (0.054) (0.057) (0.054) (0.039) (0.060) Judge and skater reside in same region 0.208*** 0.222*** 0.017 0.004 0.117*** 0.122*** (0.063) (0.065) (0.018) (0.018) (0.037) (0.039) Long Programme −0.055** −0.057** −0.069* −0.069* 0.013 0.013 (0.026) (0.026) (0.040) (0.040) (0.065) (0.065) Same sex judge and skater −0.047 0.004 0.133*** (0.040) (0.039) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 27,665 27,665 27,665 12,385 12,385 12,385 7,830 7,830 7,830 R-squared 0.777 0.704 0.704 0.771 0.707 0.707 0.848 0.813 0.813 Regional Sectional National (same as table 7) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.190*** 0.172*** 0.172*** 0.100* 0.104* 0.101* 0.127** 0.207*** 0.110* (0.028) (0.039) (0.039) (0.054) (0.054) (0.057) (0.054) (0.039) (0.060) Judge and skater reside in same region 0.208*** 0.222*** 0.017 0.004 0.117*** 0.122*** (0.063) (0.065) (0.018) (0.018) (0.037) (0.039) Long Programme −0.055** −0.057** −0.069* −0.069* 0.013 0.013 (0.026) (0.026) (0.040) (0.040) (0.065) (0.065) Same sex judge and skater −0.047 0.004 0.133*** (0.040) (0.039) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 27,665 27,665 27,665 12,385 12,385 12,385 7,830 7,830 7,830 R-squared 0.777 0.704 0.704 0.771 0.707 0.707 0.848 0.813 0.813 Source: Authors’ calculations. Note: Dependent variable is the mark a judge assigns for a programme component. The data comprises 13 competitions. The regional competitions included are the CP Regional 2012, EGL Regional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, SA Regional 2012, SW Regional 2012, SWP Regional 2012, and UGL Regional 2012. The sectional competitions are the Eastern Sectional 2012, Midwestern Sectional 2012, Pacific Coast Sectional 2012. The last three columns are based on the National Competition 2012. Standard errors are clustered by competitor and judge. *** p < 0.01, ** p < 0.05, * p < 0.1, two-tailed test. Table 6 The effect of shared club affiliation on programme component marks: all levels of qualifying competitions, singles only, 2011–12 Regional Sectional National (same as table 7) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.190*** 0.172*** 0.172*** 0.100* 0.104* 0.101* 0.127** 0.207*** 0.110* (0.028) (0.039) (0.039) (0.054) (0.054) (0.057) (0.054) (0.039) (0.060) Judge and skater reside in same region 0.208*** 0.222*** 0.017 0.004 0.117*** 0.122*** (0.063) (0.065) (0.018) (0.018) (0.037) (0.039) Long Programme −0.055** −0.057** −0.069* −0.069* 0.013 0.013 (0.026) (0.026) (0.040) (0.040) (0.065) (0.065) Same sex judge and skater −0.047 0.004 0.133*** (0.040) (0.039) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 27,665 27,665 27,665 12,385 12,385 12,385 7,830 7,830 7,830 R-squared 0.777 0.704 0.704 0.771 0.707 0.707 0.848 0.813 0.813 Regional Sectional National (same as table 7) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.190*** 0.172*** 0.172*** 0.100* 0.104* 0.101* 0.127** 0.207*** 0.110* (0.028) (0.039) (0.039) (0.054) (0.054) (0.057) (0.054) (0.039) (0.060) Judge and skater reside in same region 0.208*** 0.222*** 0.017 0.004 0.117*** 0.122*** (0.063) (0.065) (0.018) (0.018) (0.037) (0.039) Long Programme −0.055** −0.057** −0.069* −0.069* 0.013 0.013 (0.026) (0.026) (0.040) (0.040) (0.065) (0.065) Same sex judge and skater −0.047 0.004 0.133*** (0.040) (0.039) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 27,665 27,665 27,665 12,385 12,385 12,385 7,830 7,830 7,830 R-squared 0.777 0.704 0.704 0.771 0.707 0.707 0.848 0.813 0.813 Source: Authors’ calculations. Note: Dependent variable is the mark a judge assigns for a programme component. The data comprises 13 competitions. The regional competitions included are the CP Regional 2012, EGL Regional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, SA Regional 2012, SW Regional 2012, SWP Regional 2012, and UGL Regional 2012. The sectional competitions are the Eastern Sectional 2012, Midwestern Sectional 2012, Pacific Coast Sectional 2012. The last three columns are based on the National Competition 2012. Standard errors are clustered by competitor and judge. *** p < 0.01, ** p < 0.05, * p < 0.1, two-tailed test. The weaker skaters are eliminated through the qualifying process and the average skating quality increases as the season progresses from regional to sectional to national competition. Meanwhile, while national, sectional and regional judges serve as officials and award marks at regional events, only national judges participate in the national event and only sectional and national judges participate in the sectional event. We find a positive and statistically significant coefficient on same-club affiliation for all levels of competition, showing that marks increase when the skater and the judge share the same club affiliation. The point estimates for the regional, sectional and national competitions are between 0.10 and 0.21. Judges share an affiliation for 6.6% of the marks in the regional competitions, 2.1% at sectional competitions, and 2.7% at the national competitions. In many of the specifications, judges who reside in the same region as a skater increase the mark awarded on top of the shared club affiliation. Because shared region is almost always concurrent with a shared club affiliation this may compound the escalation of marks. There is some variation in the point estimates of the club affiliation variable for different levels of competition, this finding could either be due to a more heterogeneous judging pool in lower level competitions, or due to the fact that at higher level competitions, judges have a stronger incentive to mark accurately, due to the increased publicity associated with higher level competitions. To distinguish between these two explanations, in Table 7 we re-estimate the specification of six for national judges.9 Only national judges can serve at regional, sectional and national competitions. We drop any marks from the regional and sectional competitions in which there were fewer than three national judges on the official panel. We find that national judges display bias from shared club affiliation in all levels of competition. For all competitions, the 95% confidence intervals of the estimates on same-club affiliation overlap. Table 7 The effect of shared club affiliation on programme component marks. NATIONAL JUDGES ONLY—all levels of qualifying competitions, singles only, 2011–12 Regional Sectional National (same as Table 6) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.121*** 0.145*** 0.145*** 0.099* 0.114** 0.108* 0.127** 0.207*** 0.110* (0.037) (0.056) (0.056) (0.059) (0.057) (0.060) (0.054) (0.039) (0.060) Judge and skater reside in same Region 0.309** 0.057 0.015 0.008 0.117*** 0.122*** (0.156) (0.072) (0.022) (0.022) (0.037) (0.039) Long Programme −0.057 −0.057 −0.082* −0.082* 0.013 0.013 (0.038) (0.039) (0.043) (0.043) (0.065) (0.065) Same sex judge and skater −0.057 0.023 0.133*** (0.063) (0.041) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 12,190 12,190 12.190 9,415 9,415 9,415 7,830 7,830 7,830 R-squared 0.817 0.736 0.736 0.776 0.707 0.707 0.848 0.813 0.813 Regional Sectional National (same as Table 6) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.121*** 0.145*** 0.145*** 0.099* 0.114** 0.108* 0.127** 0.207*** 0.110* (0.037) (0.056) (0.056) (0.059) (0.057) (0.060) (0.054) (0.039) (0.060) Judge and skater reside in same Region 0.309** 0.057 0.015 0.008 0.117*** 0.122*** (0.156) (0.072) (0.022) (0.022) (0.037) (0.039) Long Programme −0.057 −0.057 −0.082* −0.082* 0.013 0.013 (0.038) (0.039) (0.043) (0.043) (0.065) (0.065) Same sex judge and skater −0.057 0.023 0.133*** (0.063) (0.041) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 12,190 12,190 12.190 9,415 9,415 9,415 7,830 7,830 7,830 R-squared 0.817 0.736 0.736 0.776 0.707 0.707 0.848 0.813 0.813 Source: Authors’ calculations. Note: Dependent variable is the mark a judge assigns for a programme component. The data comprises 13 competitions. The last three columns are based on the National Championships 2012 and are identical to columns 7–9 in Table 6. For columns 1–6, panels consisting of less than three national level judges at the regional and sectional competitions have been dropped. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Table 7 The effect of shared club affiliation on programme component marks. NATIONAL JUDGES ONLY—all levels of qualifying competitions, singles only, 2011–12 Regional Sectional National (same as Table 6) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.121*** 0.145*** 0.145*** 0.099* 0.114** 0.108* 0.127** 0.207*** 0.110* (0.037) (0.056) (0.056) (0.059) (0.057) (0.060) (0.054) (0.039) (0.060) Judge and skater reside in same Region 0.309** 0.057 0.015 0.008 0.117*** 0.122*** (0.156) (0.072) (0.022) (0.022) (0.037) (0.039) Long Programme −0.057 −0.057 −0.082* −0.082* 0.013 0.013 (0.038) (0.039) (0.043) (0.043) (0.065) (0.065) Same sex judge and skater −0.057 0.023 0.133*** (0.063) (0.041) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 12,190 12,190 12.190 9,415 9,415 9,415 7,830 7,830 7,830 R-squared 0.817 0.736 0.736 0.776 0.707 0.707 0.848 0.813 0.813 Regional Sectional National (same as Table 6) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.121*** 0.145*** 0.145*** 0.099* 0.114** 0.108* 0.127** 0.207*** 0.110* (0.037) (0.056) (0.056) (0.059) (0.057) (0.060) (0.054) (0.039) (0.060) Judge and skater reside in same Region 0.309** 0.057 0.015 0.008 0.117*** 0.122*** (0.156) (0.072) (0.022) (0.022) (0.037) (0.039) Long Programme −0.057 −0.057 −0.082* −0.082* 0.013 0.013 (0.038) (0.039) (0.043) (0.043) (0.065) (0.065) Same sex judge and skater −0.057 0.023 0.133*** (0.063) (0.041) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 12,190 12,190 12.190 9,415 9,415 9,415 7,830 7,830 7,830 R-squared 0.817 0.736 0.736 0.776 0.707 0.707 0.848 0.813 0.813 Source: Authors’ calculations. Note: Dependent variable is the mark a judge assigns for a programme component. The data comprises 13 competitions. The last three columns are based on the National Championships 2012 and are identical to columns 7–9 in Table 6. For columns 1–6, panels consisting of less than three national level judges at the regional and sectional competitions have been dropped. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. 5.4 Grade of Execution vs component marks For the technical scores, each judge awards a GOE mark for each element in a programme. The number of elements depends on the skater or team skill level, and on whether it is a free skating or short programme. For each element, GOE marks range from -3 to + 3 in increments of 1. Our unit of analysis for the technical marking is the individual GOE mark given by a judge for a given skater for a single element in a programme. We test the hypothesis that the evaluation of programme component marks is more biased than the evaluation of GOE marks. Two institutional features motivate this hypothesis. First, with GOE marks judges have less discretion. The guidelines for awarding GOE marks have very specific directions for a base GOE mark with deductions and reductions for errors and increases for positive aspects. Second, for GOE marks, an increase of one increment is equivalent to a 0.728 standard deviation, while if the component mark is increased by one increment, the result is a 0.2 standard deviation increase. Because a GOE one increment increase is larger in terms of standard deviation, a one increment increase in a GOE mark is more likely to be noticed than an increment increase in component marks. The first four columns of Table 8 present the regression results for the GOE marks for singles competitions for the 2011–12 season.10 For the last four columns reproduce the analog specifications for component marks that we had presented in Table 3A. In each of the first four columns, we find that the point estimate on same club affiliation is less for the GOE marks than for the component marks. The estimates for the GOE marks range from 0.027 to 0.038 while the programme component marks range from 0.141 to 0.164. Thus, judges appear to be more objective and show less favouritism in awarding marks when they have less discretion. Table 8 Comparison of Grade of Execution marks vs component marks: all levels of qualifying competitions, singles only, 2011–12 Dependent Variable GOE Programme Component (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.038** 0.027* 0.037* 0.034* 0.164*** 0.160*** 0.141*** 0.152*** (0.015) (0.016) (0.013) (0.016) (0.025) (0.026) (0.034) (0.034) Judge and skater reside in same region 0.029** 0.031* 0.021 (0.012) (0.019) (0.018) (0.017) Log distance between skater club and judge’s residence −0.010** −0.006 0.006 −0.011 (0.004) (0.007) (0.006) (0.013) Long Programme 0.034 0.034 −0.046** −0.046** (0.028) (0.028) (0.021) (0.021) Same sex judge and skater −0.005 −0.004 0.022 0.022 (0.012) (0.012) (0.026) (0.026) Performance Fixed Effect? Yes Yes No No Yes Yes No No Skater Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No Yes Yes No No Yes Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Observations 84,309 84,309 84,309 84,309 47,880 47,880 47,880 47,880 R-squared 0.204 0.204 0.150 0.150 0.848 0.848 0.779 0.779 Dependent Variable GOE Programme Component (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.038** 0.027* 0.037* 0.034* 0.164*** 0.160*** 0.141*** 0.152*** (0.015) (0.016) (0.013) (0.016) (0.025) (0.026) (0.034) (0.034) Judge and skater reside in same region 0.029** 0.031* 0.021 (0.012) (0.019) (0.018) (0.017) Log distance between skater club and judge’s residence −0.010** −0.006 0.006 −0.011 (0.004) (0.007) (0.006) (0.013) Long Programme 0.034 0.034 −0.046** −0.046** (0.028) (0.028) (0.021) (0.021) Same sex judge and skater −0.005 −0.004 0.022 0.022 (0.012) (0.012) (0.026) (0.026) Performance Fixed Effect? Yes Yes No No Yes Yes No No Skater Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No Yes Yes No No Yes Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Observations 84,309 84,309 84,309 84,309 47,880 47,880 47,880 47,880 R-squared 0.204 0.204 0.150 0.150 0.848 0.848 0.779 0.779 Source: Authors’ calculations. Note: Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Table 8 Comparison of Grade of Execution marks vs component marks: all levels of qualifying competitions, singles only, 2011–12 Dependent Variable GOE Programme Component (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.038** 0.027* 0.037* 0.034* 0.164*** 0.160*** 0.141*** 0.152*** (0.015) (0.016) (0.013) (0.016) (0.025) (0.026) (0.034) (0.034) Judge and skater reside in same region 0.029** 0.031* 0.021 (0.012) (0.019) (0.018) (0.017) Log distance between skater club and judge’s residence −0.010** −0.006 0.006 −0.011 (0.004) (0.007) (0.006) (0.013) Long Programme 0.034 0.034 −0.046** −0.046** (0.028) (0.028) (0.021) (0.021) Same sex judge and skater −0.005 −0.004 0.022 0.022 (0.012) (0.012) (0.026) (0.026) Performance Fixed Effect? Yes Yes No No Yes Yes No No Skater Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No Yes Yes No No Yes Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Observations 84,309 84,309 84,309 84,309 47,880 47,880 47,880 47,880 R-squared 0.204 0.204 0.150 0.150 0.848 0.848 0.779 0.779 Dependent Variable GOE Programme Component (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.038** 0.027* 0.037* 0.034* 0.164*** 0.160*** 0.141*** 0.152*** (0.015) (0.016) (0.013) (0.016) (0.025) (0.026) (0.034) (0.034) Judge and skater reside in same region 0.029** 0.031* 0.021 (0.012) (0.019) (0.018) (0.017) Log distance between skater club and judge’s residence −0.010** −0.006 0.006 −0.011 (0.004) (0.007) (0.006) (0.013) Long Programme 0.034 0.034 −0.046** −0.046** (0.028) (0.028) (0.021) (0.021) Same sex judge and skater −0.005 −0.004 0.022 0.022 (0.012) (0.012) (0.026) (0.026) Performance Fixed Effect? Yes Yes No No Yes Yes No No Skater Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No Yes Yes No No Yes Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Observations 84,309 84,309 84,309 84,309 47,880 47,880 47,880 47,880 R-squared 0.204 0.204 0.150 0.150 0.848 0.848 0.779 0.779 Source: Authors’ calculations. Note: Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. 6. Conclusion Using data from figure skating competitions in the USA, we study bias in judging decisions and find that judges show favouritism in awarding higher marks to skaters with whom they share the same club affiliation. This finding, that is, that shared group identity is a determinant of judging decisions, is consistent with theoretical work on identity and individual decision-making. We document that judges make biased decisions due to shared membership in the club with the skater, even when controlling for physical proximity as measured by the distance between judges’ homes and skaters’ clubs. Thus, as a social construct, a club often elicits greater affiliation bias than bias due to physical proximity. We also provide evidence that judges may act strategically. An additional finding is that decisions that allow for greater judging discretion are subject to larger bias than decisions subject to stricter guidelines. These results are relevant to many contexts when designing rule structures with the objective to limit the discretion of the evaluator so as to decrease the impact of bias. While group identity of a skating club is not as meaningful or salient as race, sex or nationality, our findings for this less salient attribute are still are consistent with bias. The impact of group membership on the evaluation of performance is relevant for the design of institutions for decision-making in hiring and promotion processes, in political and legal systems and other non-sports settings. However, as in many other settings, one of the difficulties in designing such institutions, as in figure skating, is the constraint that the pool of individuals that are qualified and willing to serve as evaluators is limited. For figure skating, the pool of judges consists of individuals who are deeply involved in the sport, are passionate about skating and who have longstanding personal relationships with others in the skating community. In corporate settings, businesses often attempt to broaden board rooms or employees by diversifying by race, gender and other, easily observable group identities. However, in contrast to diversifying with respect to the latter characteristics, it is much more complex to diversify with respect to type of group of identity that has a low salience, similar to skating club memberships. Our findings suggest that identifying and perhaps screening for less visible group memberships might be appropriate in settings in which there are incentives for individuals to show favouritism for in-group members. To date, figure skating rules do include conflict of interest policies for judges. In this sport, conflict of interest rules are solely limited to family members and those with financial relationships with skaters. Ceteris paribus, our results suggest that implementing of stricter conflict of interest policies, based on same club membership is predicted to reduce biased judging in figure skating. From personal interviews with judges, we learned that skating judges do not believe that they favour members of their club. Instead, they report that they tend to judge club members more harshly. One potential explanation to explain this apparent contradiction between our results and their beliefs, is that skating judges increase their marks to compensate for their belief that they judge skaters from their club too harshly. In this situation, additional information has the potential to reduce biased judging. Mere documentation of bias might raise judges’ awareness and can give them a sufficient incentive to change their behaviour. One example supporting this conjecture is that racial bias by officials in the National Basketball Association was reduced not through institutional changes but seemingly by making basketball referees aware of their own bias (Pope et al., 2013). The findings in this paper provide some support to the view that bias-reduction can be achieved through the selection of a pool of evaluators who are somewhat removed, in terms of group identity, from the person to be judged. One limitation of our data set is that we can only recognize and account for transparent relationships between skaters and judges. Over time, many individual skaters change locations to train. Some skaters retain membership in their initial club while others change membership. Judges, also, relocate and change club affiliations for work and family reasons. Because we can only identify the most transparent relationships, our empirical analysis may only identify the lower bound of any judging bias. This feature of our data resembles other settings in society in which objective evaluation is expected, such as from hiring committees and job applicants, or judges and attorneys, where potential shared group memberships may be largely invisible. Supplementary material The Appendix and Data files are available online at the OUP website. Footnotes 1 In 2016, the International Skating Union, with the support of the US Figure Skating Association changed the rules for international competition to mandate the identification of judges and marks awarded in international competitions including future Olympic and World Championship competitions. This rule change will allow for future studies on nationalism in figure skating judging. 2 For our sample period, juvenile and intermediate skaters do not compete in sectionals, but compete at a junior national event. 3 See http://usfigureskating.org/content/2014-15%20Rulebook%2008-14-14.pdf- (accessed 1 March 2015). 4 It is possible that a skater and judge shared a club affiliation prior to the competitions, we consider this, but no longer apply it to our data set, because the judge now belongs to a different club. We do not observe such past affiliations. To the extent that such past affiliations exist for some of our observations, unidentified relationships tend to increase favouritism, so we can view our estimates as a lower bound. 5 Given that there are five component marks, without any ties, only 20% of the marks would be the highest mark. However, because of ties there can be two highest values, and thus the percentage of high marks is greater than 20%. 6 The official ISU’s documentation explains that a component mark of 5 connotes average performance of the criteria, with a 4 being fair. This means that the average programme observed in the US qualifying stream is below average. 7 Summary statistics for Table 4 are presented in the Appendix Table A2. 8 Appendix Table A3 presents the summary statistics for Table 5. 9 We report the summary statistics for the variables of Table 6 in Appendix Table A4 while the variables of Table 7 can be found in Appendix Table A5. 10 Appendix Table A6 contains the summary statistics for Table 8 References 2011–12 Rulebook.pdf. Available at: http://usfsa.org/Content/201112Rulebook.pdf (accessed 1 March 2015). Adams M. ( 2011 ) Artistic Impressions: Figure Skating, Masculinity, and the Limits of Sport , University of Toronto Press , Toronto . Akerlof G. , Kranton R. ( 2000 ) Economics and identity , The Quarterly Journal of Economics , 115 , 715 – 53 . Google Scholar CrossRef Search ADS Bar T. , Zussman A. ( 2012 ) Partisan grading , American Economic Journal: Applied Economics , 4 , 30 – 48 . Google Scholar CrossRef Search ADS Bertrand M. , Chugh D. , Mullainathan S. ( 2005 ) Implicit discrimination , The American Economic Review , 95 , 94 – 8 . Google Scholar CrossRef Search ADS Charness G. , Rigotti L. , Rustichini A. ( 2007 ) Individual behavior and group membership, American Economic Review , 97 , 1340 – 52 . Google Scholar CrossRef Search ADS Chen Y. , Li S. ( 2009 ) Group identity and social preferences , The American Economic Review , 99 , 431 – 57 . Google Scholar CrossRef Search ADS Emerson J. , Arnold T. ( 2011 ) Statistical sleuthing by leveraging human nature: a study of Olympic figure skating, The American Statistician , 65 , 143 – 8 . Google Scholar CrossRef Search ADS Emerson J. , Meredith S. ( 2011 ) Nationalistic judging bias in the 2000 Olympic diving competition , Math Horizons , 18 , 8 – 11 . Google Scholar CrossRef Search ADS Emerson J. , Seltzer M. , Lin D. ( 2009 ) Assessing judging bias: an example from the 2000 Olympic games , The American Statistician , 63 , 124 – 31 . Google Scholar CrossRef Search ADS Goette L. , Huffman D. , Meier S. ( 2006 ) The impact of group membership on cooperation and norm enforcement: evidence using random assignment to real social groups , The American Economic Review , 96 , 212 – 6 . Google Scholar CrossRef Search ADS Kestnbaum E. ( 2003 ) Culture on Ice: Figure Skating and Cultural Meaning , Wesleyan University Press , Middletown, CT . Kim J. , King B. ( 2014 ) Seeing stars: matthew effects and status bias in major league baseball umpiring , Management Science , 60 , 2619 – 44 . Google Scholar CrossRef Search ADS Leskovsek B. , Cuk I. , Pajek J. , Forbes W. , Buvcar-Pajek M. ( 2012 ) Bias of judging in men’s artistic gymnastics at the European championship 2011 , Biology of Sport , 29 , 107 – 13 . Google Scholar CrossRef Search ADS Looney M. ( 2012 ) Judging anomalies at the 2010 Olympics in men’s figure skating , Measurement in Physical Education and Exercise Science , 16 , 55 – 68 . Google Scholar CrossRef Search ADS Parsons C. , Sulaeman J. , Yates M. , Hamermesh D. ( 2011 ) Strike three: discrimination, incentives, and evaluation, American Economic Review , 101 , 1410 – 35 . Google Scholar CrossRef Search ADS Pope D. , Price J. , Wolfers J. ( 2013 ) Awareness Reduces Racial Bias (Working Paper No. 19765), National Bureau of Economic Research, Cambridge, MA. Popović R. ( 2000 ) International bias detected in judging rhythmic gymnastics competition at Sydney 2000 Olympic Games , Facta Universitatis-Series: Physical Education and Sport , 1 , 1 – 13 . Price J. , Wolfers J. ( 2010 ) Racial discrimination among NBA referees, Quarterly Journal of Economics , 125 , 1859 – 87 . Google Scholar CrossRef Search ADS Shayo M. , Zussman A. ( 2011 ) Judicial ingroup bias in the shadow of terrorism , The Quarterly Journal of Economics , 126 , 1447 – 84 . Google Scholar CrossRef Search ADS Tajfel H. ( 1970 ) Experiments in intergroup discrimination, Scientific American , 223 , 96 – 102 . Google Scholar CrossRef Search ADS PubMed Whissell R. , Lyons S. , Wilkinson D. , Whissell C. ( 1993 ) National bias in judgements of Olympic-level skating, Perceptual and Motor Skills , 77 , 355 – 8 . Google Scholar CrossRef Search ADS Zitzewitz E. ( 2006 ) Nationalism in winter sports judging and its lessons for organizational decision making, Journal of Economics and Management Strategy , 15 , 67 – 99 . Google Scholar CrossRef Search ADS Zitzewitz E. ( 2014 ) Does transparency reduce favoritism and corruption? Evidence from the reform of figure skating judging , Journal of Sports Economics , 15 , 3 – 30 . Google Scholar CrossRef Search ADS © Oxford University Press 2018 All rights reserved This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Oxford Economic Papers Oxford University Press

Judging on thin ice: the effects of group membership on evaluation

Loading next page...
 
/lp/ou_press/judging-on-thin-ice-the-effects-of-group-membership-on-evaluation-5LPlLQEquW
Publisher
Oxford University Press
Copyright
© Oxford University Press 2018 All rights reserved
ISSN
0030-7653
eISSN
1464-3812
D.O.I.
10.1093/oep/gpx054
Publisher site
See Article on Publisher Site

Abstract

Abstract Employment, legal and social settings often expect the objective evaluation of others regardless of race, sex or other group identification. Such objectivity may be undermined by group membership that is difficult to detect. In this paper we exploit an institutional feature of figure skating, namely that figure skaters possess individual styles and technique, rather than their performance being influenced by club-specific characteristics, which allows us to identify the role of favouritism in a context where it is intended to be excluded. We can identify judges being influenced by group identity, in this case club membership and exhibiting favouritism for club members. We discuss the implications of our findings for rule which would help limit the role of favouritism in decision-making. 1. Introduction Rules often mandate objectivity and fairness from those who evaluate others. Impartiality is imperative for objective evaluation of legal issues, employment and other settings, including sports. When evaluating the performance or productivity of competing individuals that has multiple dimensions and requires quick and complex decisions, attributes different from the actual conduct or performance, such as race, gender, religion, or other group identifications may influence assessments. Some group identities, such as those shared by university alumni, fraternity and sorority members, or civic associations such as Masons or Elks clubs, are not always apparent to non-group members yet may also influence evaluators who share group membership. The lack of visibility of those affiliations frustrates attempts to study situations where one might suspect bias and may allow favouritism in many settings to go unnoticed and unstudied. Corporate settings often use evaluation either by one individual or by a committee to make decisions regarding the hiring and advancement of others. Visible group memberships such as race and gender may be addressed through diversity in the decision-making body. However other and less visible group identities cannot be as easily addressed. Well-known work in social psychology analyses group membership in an experimental setting. This work shows that even seemingly meaningless distinctions among groups can lead individuals to favour those within their group relative to those outside of their group (Tajfel, 1970). Economists have contributed to this area of study with theoretical and experimental analyses of the effect of identity and of in-group membership (Akerlof and Kranton, 2000; Bertrand et al., 2005; Goette et al., 2006; Charness et al, 2007; Chen and Li, 2009; Shayo and Zussman, 2011; Bar and Zussman, 2012). Random assignment into real social groups in Goette et al. (2006) provides stronger group identification than mere laboratory artificial groups and fosters greater cooperation rates within groups. In this paper, we analyse how mandatory yet self-chosen group affiliation impacts behaviour. In the sports economics literature, there is an abundance of work on favouritism due to race, gender and ethnicity. This literature includes work showing that individuals are more likely to favourably evaluate those who share the same race (Price and Wolfers, 2010; Parsons et al., 2011). Biased judging due to nationalism is emphasized in analyses of Olympic sports such as diving (Emerson et al., 2009; Emerson and Arnold, 2011), ski jumping and figure skating (Zitzewitz, 2006) and gymnastics (Leskovsek et al., 2012). Olympic figure skating marks using an older scoring system were also found to be subject to national bias (Whissell et al., 1993). Recent studies analyse judging anomalies in the sport of figure skating with a focus on Olympic competitions in which the judges’ marks have been anonymous since 2005 (Looney, 2012; Zitzewitz, 2014).1 A recent study found that umpires in Major League Baseball tended to expand the strike zone in favour of the home team (Kim and King, 2014). This is especially interesting as the umpires are in an environment in which every ball and strike call can be evaluated by impartial technology. In this study, we have identified a potential source of biased judging for skating competition which is the institution of club membership in US Figure Skating. Membership of both skaters and judges in a local skating club is required. One advantage of this setting and the corresponding data is that in contrast to international anonymity, data from US figure skating competitions allows us to identify the club membership of both the skater and skating judges, along with the details of the skating performance and evaluation. The structure of figure skating in the USA provides us with the opportunity to empirically separate characteristics of performance from other characteristics that might result in biased judging. In the USA officials evaluate skating performances according to the International Judging System (IJS), which is a set of objective criteria that was developed by the International Skating Union (ISU). Because judges are randomly assigned to competitions and events in the qualifying competitions up to and including the National Championships, we are not concerned with selection issues that might bias our results if judges could choose to be on the judging panels for skaters that they favour. One benefit of our data is that there is a panel of judges, rather than just one, each of whom performs their evaluation independently, evaluating each skater’s performance. This allows us to control for the quality of the skate, as judged by peers. We show that skaters receive higher marks from judges with whom they share a club affiliation. We find these results for singles skating, senior and non-senior levels of skating ability, and national, sectional and regional events. We also find support for the hypothesis that judges follow a strategy of showing favouritism in cases where the results matter the most, at the point of discontinuity, when skaters are on the verge of elimination from advancement to higher-level competitions. Additionally, we examine evaluation decisions when the guidelines for awarding marks are less strict. Here, we find a larger bias when judges have greater discretion in awarding marks. One concern regarding our analysis might be that skating styles differ by clubs, and that skating judges favour skaters in their club, because they prefer a particular club’s skating style. However, evidence suggests that skaters from different skating clubs do not differ systematically with respect to their style of skating but that individual characteristics and individual style matter in skating. According to Ellyn Kestnbaum’s Culture on Ice: Figure Skating and Cultural Meaning, figure skaters bring their individual strengths and weaknesses to their performances. A skater’s most outstanding abilities could be those of being more athletic, technical or artistic. Judges reward excellence of any kind when they see it. ‘No mathematical formula can capture the entirety of a skating performance. The best ones are designed not simply to showcase elements. … The best skating programs are meant to move us’ (Adams, 2011, p.75). 2. Institutions: skating clubs, skating disciplines, qualifying structure and judges 2.1 Skating clubs Skaters, officials and coaches must all be members of US Figure Skating through a local club to participate in skating competitions. We rely on the assumption, supported by anecdotal evidence, that clubs are not known for skating styles, but for the amenities they offer to skaters and their families and for their geographic locations. The style of a skater is based on individual strengths and weaknesses in combination with personal choices by the skater as to their music, choreographer and techniques. Coaches, parents and other members of a skater's individual ‘team’ assist in decisions regarding the design of choreography, costumes, make-up and hairstyles, etc. When groups maintain specific characteristics, it is difficult to determine whether favouritism in evaluations is due to bias, or due to placing a higher weight on same-group characteristics. Anecdotal evidence suggests that figure skating club membership does not connote any specific same-group similarities (Kestnbaum, 2003). Though there are some locations in which skaters cluster to train with other high calibre, internationally competitive skaters, skaters usually compete for their home club rather than changing their club affiliation to their training location. 2.2 Disciplines, competitions, events and divisions Competitive figure skating has five disciplines: ladies’ singles, men's singles, synchronized team skating, pairs and ice dancing. Singles skaters and teams have unique club affiliations, allowing us to study potential bias in evaluation. In contrast, pair skaters and members of ice-dance teams may represent different clubs, or are located in different parts of the country. Hence, for the latter disciplines, skaters do not always share a group membership, and this is why we exclude pairs and ice-dancing competitions from our analysis. For each figure skating discipline in this study—ladies’ singles and men's singles—a competition has events for skaters of different skill levels. Singles competitors have one of five skill levels. These skill levels are, in order of technical proficiency, juvenile, intermediate, novice, junior and senior. Higher skill levels require the performance of more difficult elements. Each skill level is subject to different rules and guidelines. A so-called event in ice skating includes in three dimensions: a competition level, a skill level and the skater’s sex. The USA is divided into nine skating regions. Each holds a regional competition, with events that are open to singles skaters with the appropriate skill levels. The top four skaters from each event in these regional competitions advance to one of the three sectional singles competitions.2 And, within each sectional event, the top four skaters or teams in each of the sectional competitions advance to the annual US National Championship. Approximately 16,000 figure skaters compete in regional competitions each year and about 175 figure skaters participate in the US National Championships. However, for ladies’ and men’s singles, in the years included in this study, only novice, junior and senior skaters compete in the US Championships. Therefore, for singles skaters, we consider only these three skill levels. 2.3 Singles skating Novice, junior and senior singles events have two separate segments: a short programme, skated first and a free skating programme, also referred to as the long programme. The short programme has required moves, referred to as elements and are determined by US Figure Skating for the intermediate skill level or by the ISU for the novice through senior skill levels. A random draw determines the skating order in the short programme—that is, who skates first, second and so on. For the free skating programme, a seeded draw determines the skating order. Here, skaters are grouped in the inverse order of their rank after the short programme. No rule determines who advances from National Championships to international championships. A US Figure Skating committee decides who of the top skaters at the National Championship will represent the USA at the next year’s international competitions. This includes the opportunity to compete at the World Championships, which awards cash prizes totaling over $700,000, with $45,000 being awarded to the men’s and ladies’ event winners. 2.4 Skating officials and scoring Judges award marks at competitions. Along with the judges, all competitions include a referee, who keeps time, makes sure other officials are at their pre-assigned seats, who stops a performance if equipment malfunctions, etc. All competitions also include a technical panel, which identifies and assigns a level of difficulty to elements. At regional, sectional and national competitions, judges evaluate the skaters in accordance with the guidelines of the IJS. Subsequent to the figure skating judging scandals in the 1998 and 2002 Olympics, the ISU introduced this system in 2004. Prior to 2004, judges awarded ordinal marks for each skater, and final placements were determined by the majority of ordinals. Using the IJS, the technical elements are identified by a technical panel and communicated to the judges as they are performed. For the evaluation of the technical elements, judges have very specific guidelines for markings, with suggested reductions and increases based on performance quality, and mandatory caps on the highest mark when major errors occur. Judges award the technical element marks, also referred to as Grade of Execution (GOE) scores while skaters perform, and award programme component marks directly after each skater or team’s performance. For the programme component score, each judge assigns a mark to each of the five individual components, which are skating skills, transitions, performance/execution, choreography and interpretation. Judges mark each programme component on a scale from 0.25 to 10.0, in increments of 0.25. Table A1 in the Online Appendix shows the computation of skaters’ overall scores. Further details on how scores are calculated is provided in the notes to Table A1. Three competency levels reflect the qualification of judges. For singles skating, the levels are regional, sectional and national. A national appointment indicates a higher level of skill and experience. Judges of all of three competency levels serve in regional competitions. Regional judges cannot serve at sectional or national events and sectional judges cannot serve at national events. Each skater’s club affiliation is announced at competitions just prior to their performance. So, it is implausible that a judge would be unaware that they share a club affiliation with a skater. Since the beginning of the US Figure Skating Association, about 90 years ago, judges have been required to become members through a membership in a local club. Judges receive no pay for their services. However, the clubs reimburse judges for judging-related expenses, including the costs of travel. To avoid a conflict of interest, rules do not allow judges to earn income by serving as a coach, choreographer, or consultant to skaters. The Judges’ Creed Standard of Conduct, which includes the statement ‘I shall make my judgment to the best of my ability with all humility and then shall keep my own counsel unless questioned officially’ (US Figure Skating Rulebook 2012, Section JR1.01, p. 67), is meant to guide the behaviour of judges.3 During the entire event, skating rules do not permit judges to discuss their evaluations among themselves. 3. Model and hypotheses We assume that judges show bias in their evaluation of a skater if the utility of having a member of their group place higher in a competition exceeds the cost of exercising such a preference. Based on this simple assumption, we generate multiple hypotheses regarding how judges evaluate skaters. Hypothesis 1 Judges will increase their marks when they and the skater belong to the same skating club. Testing for biased evaluation based on a shared group membership is different from testing for bias based on race or gender. For race or gender, favouritism may be due to heritable shared genetics rather than a choice of affiliation. In contrast, clubs are social constructs and individuals choose to belong to a particular club. We hypothesize that a social construct, club affiliation, gives rise to biased decision-making. Biased evaluation might also originate from geographical location. That is, judges may evaluate skaters more favourably when the distance between their residence and that of the skater’s club is less. Biased judging based on geography might also occur based on whether the judge’s residence is located in the athlete’s skating region, as defined by US Figure Skating Our first hypothesis predicts that judges will award higher marks to skaters who share the same club affiliation as the judge. We refer to the higher marking by a judge as affiliation bias, when this judge has the same the same club affiliation as the skater. Hypothesis 2 Affiliation bias in evaluation increases when a skater is on the cusp of elimination from further competitions. The top placed four males and four females from each skill level (novice, junior and senior) advance from the regional qualifying competition to the sectional and from the sectional to the national competition. Fifth placed skaters do not advance to the next level of competition. This non-linearity in the benefits of a skater’s final placement alters the cost-benefit analysis for biased judging. The short programmes are skated a day or two before the free skating programmes and the results are publicly available. The fourth-placed skater after the short programme faces the danger of not advancing. A judge who shares a club affiliation with such a skater ‘on the bubble’ has an increased incentive to show favouritism. We predict that skaters in fourth and fifth place after the short programme at regionals and sectionals competitions will receive higher marks from judges of their own club than skaters more securely at the top of the standings, or at the bottom of the standings after the short programme. Hypothesis 3 Evaluation of technical elements will be more objective than the programme component marks. Judges have much more discretion in marking components of a performance, such as choreography, than in marking the grades of execution of the technical elements. While GOE marks must be reduced for a fall or a major error, there is no corresponding requirement for programme component marks. Additionally, GOE marks have a narrower range and have larger, and thus fewer, increments than the component marks. The fact that the increments are larger for GOE marks serves as a barrier to exercising bias in awarding GOE marks, because an increment increase in GOE marks are more easily detected as being due to bias or error, than component marks that come in smaller increments. For these reasons, we predict to identify less bias in marks for the GOE of technical elements than for the marks associated with the five programme components. 4. Data sources and empirical model The institutions of figure skating allow us to examine the group biases based on a quasi-random assignment. US Figure Skating randomly assigns judges each year to each of regional and sectional competitions, as well as to the National Championships. The constraints to the random assignment are eligibility and availability of judges. There are multiple competitions with similar timeframes, implying that manipulating one’s availability is no guarantee of an assignment to a particular competition. Additionally, a judge is assigned to qualifying competitions for the season (October to January) in the prior summer, but is not assigned to specific events until just prior to the beginning of the competition. Thus, we expect no selection issues, as would be the case if judges could choose events in which affiliated skaters are participating. In addition, all judges must be members of US Figure Skating, primarily through membership in a local club. Thus, no selection bias comes from some judges choosing to have a club affiliation while others do not. Further, there is no link between the quality of skaters from a given club and the tendency of judges to belong to that club, that is, judges rarely change affiliations for reasons other than relocation. In our data no judge changed his or her club affiliation.4 In some ways, our data mimic those of a randomized experiment, however, a study like ours that uses field data certainly has shortcomings when compared to one using experimental data. In a well-designed experiment, we can attribute observed differences in behaviour solely to group assignment. This is also the goal of our empirical specification. In our design, judges who do not share club affiliation serve as a control group, allowing us to identify bias arising from shared club affiliation. We collected data from the US Figure Skating website (http://www.usfsa.org [accessed 1 March 2015]) and its affiliate Icenetwork (http://web.icenetwork.com/home [accessed 4 October 2014]). We obtained data on the official competition results for all final rounds of qualifying competitions leading to the National Championships in 2012, and for the 2010, 2011 and 2012 National Championships. Our data allow us to link each skater’s marks to the judge who awarded those marks. We identified the club affiliation of the judges by searching skating club websites, searching the Internet for information regarding their biographies and conducting personal interviews. We obtained the club affiliation of skaters and teams from official competition results. We measure distance between the judge’s residence and the club’s location by the distance between their respective ZIP codes. Information regarding judges’ home ZIP codes comes from the Directory of US Figure Skating Officials 2011–12. One approach to identify bias is to analyse simply whether a judge gives higher scores to skaters with which they share a club membership. This method will result in biased estimates if skaters from an individual judge’s club are better skaters. A second approach to test for bias is to examine the difference between the particular marks of a judge for a skater performance with a shared club affiliation, and the untrimmed mean of all marks given by other judges for the same performance. Previous work in this area uses the average of all judges’ scores as a proxy for the unobservable ‘true’ quality of the performance (Emerson and Meredith, 2011) However, when using this approach, there is a potential for endogeneity. Instead we address endogeneity issues by using indicators to control for the quality of the specific performance, the marks for each judge and other dummy variables. To test our hypotheses, our preferred specification is: markijk=β1SameClubijk+β2Locationij+γi+δj+εijk (1) where markijk is the mark given for a specific performance i by judge j for component k. We employ a similar specification to test the GOE marks, wherein markijk is the GOE mark given for perfomance i by judge j for element k. To test whether any potential favouritism arises based on club affiliation, geographical preferences, or both, we include the variable SameClubijk, which equals one if the judge shares a club affiliation with the skater or skating team. Key to our identification strategy is the assumption that skating styles vary by skater, not by club. As we have described, clubs are merely social entities that provide the right to train on a specific rink at a specific time. We are assuming that no club-specific or geography-specific skating styles exist in figure skating. The variable Locationijkt measures the geographic location of the skater and judge. We use two alternative indicators for geographic location. In one specification, we include an indicator for whether or not the judge resides in the same skating region in which the skater's club is located. In the other specification, we include the distance between the judge's residence and the skater's club location. We include performance fixed effect γi to control for a skater’s overall skill and ability and for any ‘home field’ advantage to the skater. In most specifications we also include judge fixed effects, δj, that captures heterogeneity among judges. We estimate various specifications of model (1) for singles free skating programmes and singles short programmes. We analyse both the programme component marks and the GOE marks. In some specifications, we also study biased judging based on the shared gender of skater and judge. In alternative specifications to model (1) we use indicators for competition and skater fixed effects instead of the performance indicator. The alternative includes an indicator for each of the competitions. For example, the regression includes an indicator for the New England Regional competition, for the Southwest Pacific Regional competition and so on. In most specifications we include skater fixed effects which capture a skater’s overall and ability. In the few instances when neither performance nor skater fixed effects are included, we add indicators for whether the skater is a junior or senior with novice being the omitted category. In most specifications we also include an indicator as to whether the programme is a free skate (long) or a short programme. For the programme component scores, each skater receives as many marks as there are judges for each of the five components. This generates multiple observations per skater. However, treating the multiple scores for each skater as independent is not appropriate. The marks awarded to a skater are correlated because all judges evaluate the same performance and share similar views regarding what counts as a high-quality or low-quality performance. Therefore, we cluster the standard errors in two dimensions, by competitor and by judge. For our baseline regression results, we focus on the programme component marks rather than on the GOE marks. Table 1A summarizes the number of programme component marks that judges award for both free skating and short programmes by competition type and by skater and team skill levels for the 2011–12 season. In parentheses, Table 1A also presents the number of programmes skated in each of these categories. Because judging panels are larger in national competitions than in regional or sectional competitions, there are more marks per judging panel in the national competitions. Judges awarded 47,940 programme component marks awarded during the 2011–12 regional, sectional and national competitions, after removing 245 marks for skaters who are independent members of US Figure Skating. Table 1A Cross-tabulation of programme component marks awarded: 2011–12 qualifying season Skill Levels Competition Novice Junior Senior Total (1) (2) (3) (4) (5) National Singles 2,160(48) 2,160(48) 3,510(78) 7,830(174) Sectional Singles 4,315(144) 4,200(140) 3,870(129) 12,385(413) Regional Singles 11,640(388) 8,885(301)** 7,200(240) 27,725(929) Totals 18,115(580) 15,245(489) 14,580(447) 47,940(1,516) Skill Levels Competition Novice Junior Senior Total (1) (2) (3) (4) (5) National Singles 2,160(48) 2,160(48) 3,510(78) 7,830(174) Sectional Singles 4,315(144) 4,200(140) 3,870(129) 12,385(413) Regional Singles 11,640(388) 8,885(301)** 7,200(240) 27,725(929) Totals 18,115(580) 15,245(489) 14,580(447) 47,940(1,516) Source: Authors’ calculations. Note: The unit of analysis is a judge’s programme component mark and data are for the 2011–12 skating season. Each judge awards five marks per programme per skater. Table 1A tabulates the total number of component marks awarded in each level of competition at each level of skating. In parentheses are the total numbers of long and short programmes. Regional and sectional competitions generally use a panel of six judges (**29 programmes judges with a five judge panel) while national competitions use nine judges, resulting in more marks per skater. Table 1A Cross-tabulation of programme component marks awarded: 2011–12 qualifying season Skill Levels Competition Novice Junior Senior Total (1) (2) (3) (4) (5) National Singles 2,160(48) 2,160(48) 3,510(78) 7,830(174) Sectional Singles 4,315(144) 4,200(140) 3,870(129) 12,385(413) Regional Singles 11,640(388) 8,885(301)** 7,200(240) 27,725(929) Totals 18,115(580) 15,245(489) 14,580(447) 47,940(1,516) Skill Levels Competition Novice Junior Senior Total (1) (2) (3) (4) (5) National Singles 2,160(48) 2,160(48) 3,510(78) 7,830(174) Sectional Singles 4,315(144) 4,200(140) 3,870(129) 12,385(413) Regional Singles 11,640(388) 8,885(301)** 7,200(240) 27,725(929) Totals 18,115(580) 15,245(489) 14,580(447) 47,940(1,516) Source: Authors’ calculations. Note: The unit of analysis is a judge’s programme component mark and data are for the 2011–12 skating season. Each judge awards five marks per programme per skater. Table 1A tabulates the total number of component marks awarded in each level of competition at each level of skating. In parentheses are the total numbers of long and short programmes. Regional and sectional competitions generally use a panel of six judges (**29 programmes judges with a five judge panel) while national competitions use nine judges, resulting in more marks per skater. Table 1B presents the cross tabulations for the number of programme component marks awarded for the US National Championship competitions in 2010, 2011 and 2012 for novice, junior and senior levels, broken down by short and free skating programmes. Single skaters perform in both programmes, unless they become injured. Therefore, almost half of the observations for singles are short programme marks while the other half consists of free skating programme marks. Our data for the three years of National Championships include 24,705 observations. Table 1B Cross-tabulation of programme component marks awarded: National Championships 2010, 2011 and 2012 Skill Levels Discipline Programme Length TV Novice Junior Senior Totals Singles Short No 3,240(72) 3,330(74) 5,850(130) 12,420(276) Singles Long No 3,240(72) 3,330(74) 6,570(146) Singles Long Yes 5,715(127) 5,715(127) Totals 6,480(144) 6,660(148) 11,565(257) 24,705(549) Skill Levels Discipline Programme Length TV Novice Junior Senior Totals Singles Short No 3,240(72) 3,330(74) 5,850(130) 12,420(276) Singles Long No 3,240(72) 3,330(74) 6,570(146) Singles Long Yes 5,715(127) 5,715(127) Totals 6,480(144) 6,660(148) 11,565(257) 24,705(549) Source: Authors’ calculations. Note: The unit of analysis is a judge’s programme component mark and data are for the 2010, 2011 and 2012 National Championships. Each judge awards five marks per programme per skater. Table 1B tabulates the total number of component marks awarded in each level of competition at each level of skating. In parentheses are the total numbers of long and short programmes. Table 1B Cross-tabulation of programme component marks awarded: National Championships 2010, 2011 and 2012 Skill Levels Discipline Programme Length TV Novice Junior Senior Totals Singles Short No 3,240(72) 3,330(74) 5,850(130) 12,420(276) Singles Long No 3,240(72) 3,330(74) 6,570(146) Singles Long Yes 5,715(127) 5,715(127) Totals 6,480(144) 6,660(148) 11,565(257) 24,705(549) Skill Levels Discipline Programme Length TV Novice Junior Senior Totals Singles Short No 3,240(72) 3,330(74) 5,850(130) 12,420(276) Singles Long No 3,240(72) 3,330(74) 6,570(146) Singles Long Yes 5,715(127) 5,715(127) Totals 6,480(144) 6,660(148) 11,565(257) 24,705(549) Source: Authors’ calculations. Note: The unit of analysis is a judge’s programme component mark and data are for the 2010, 2011 and 2012 National Championships. Each judge awards five marks per programme per skater. Table 1B tabulates the total number of component marks awarded in each level of competition at each level of skating. In parentheses are the total numbers of long and short programmes. 5. Results 5.1 Group membership and evaluation Starting with cross-tabulations, we first computed the number of observations with the highest mark for each component given to each skater for each programme. We then separated the marks into two groups, those in which the judge and skater share a club affiliation and those in which they do not. Table 2 shows that high marks constitute 22.41% of all marks.5 The highest marks awarded by judges to skaters with whom they share the same club affiliation constitute 29.44% of all high marks. Marks awarded by judges to skaters with whom they do not share the same club affiliation constitute 21.99% of the highest marks. We reject the hypothesis that this difference between two groups is due to chance at the 1% level (binomial p, one-tailed. = 0.00). These results provide initial evidence that judges tend to award the highest marks when they share the same club affiliation with the skater they are evaluating. Table 2 Tabulation of highest programme component mark Number of Marks Shared Club Affiliation All Highest % Highest Marks (1) (2) (3) Yes 2,711 798 29.44 No 45,169 9,936 21.99 Totals 47,880 10,734 22.41 Number of Marks Shared Club Affiliation All Highest % Highest Marks (1) (2) (3) Yes 2,711 798 29.44 No 45,169 9,936 21.99 Totals 47,880 10,734 22.41 Source: Authors’ calculations. Note: The table reports data from the 2011–12 skating season. Column (1) shows the total number of observations and the number of observations where the judge shares the same club affiliation or not. Column (2) considers only the observations that constitute the highest mark per judge per component per programme skated. Column (3) provides percentage of these marks that are given by judges with and without the same club affiliation as the skater. Table 2 Tabulation of highest programme component mark Number of Marks Shared Club Affiliation All Highest % Highest Marks (1) (2) (3) Yes 2,711 798 29.44 No 45,169 9,936 21.99 Totals 47,880 10,734 22.41 Number of Marks Shared Club Affiliation All Highest % Highest Marks (1) (2) (3) Yes 2,711 798 29.44 No 45,169 9,936 21.99 Totals 47,880 10,734 22.41 Source: Authors’ calculations. Note: The table reports data from the 2011–12 skating season. Column (1) shows the total number of observations and the number of observations where the judge shares the same club affiliation or not. Column (2) considers only the observations that constitute the highest mark per judge per component per programme skated. Column (3) provides percentage of these marks that are given by judges with and without the same club affiliation as the skater. Table 3A presents regression results for the 2011–12 season. The dependent variable in this table is the programme component mark. The observations come from all qualifying events: the nine regional singles competitions, three sectional singles competitions, and one national singles championship competition. Table 3B reports the descriptive statistics for the dependent and independent variables for the regression results in Table 3A. For the regressions in Table 3A, the mean component mark is 4.363 and the standard deviation is 1.188.6Table 3B shows that the marks awarded for skating events range from 0.75 to 9.75. The highest mark possible is 10. Table 3A The effect of shared club affiliation on programme component marks: All levels of qualifying competitions (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.136*** 0.123*** 0.164*** 0.160*** 0.255*** 0.235*** 0.141*** 0.152*** (0.032) (0.034) (0.025) (0.026) (0.054) (0.054) (0.034) (0.034) Judge and skater reside in same region 0.041 0.025 0.016 0.021 (0.027) (0.018) (0.039) (0.017) Log distance of skater club and judge’s residence 0.014* 0.006 −0.011 −0.011 (0.008) (0.006) (0.013) (0.013) Programme Length −0.022 −0.022 −0.046** −0.046** (0.023) (0.024) (0.021) (0.021) Same sex judge and skater 0.024 0.024 0.022 0.022 −0.103* −0.102* 0.018 0.018 (0.029) (0.028) (0.026) (0.026) (0.053) (0.054) (0.032) (0.032) Junior skater 0.404*** 0.404*** (0.059) (0.059) Senior skater 0.932*** 0.933*** (0.027) (0.027) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 47,880 47,880 47,880 47,880 47,880 47,880 47,880 47,880 R-squared 0.826 0.826 0.848 0.848 0.406 0.406 0.779 0.779 (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.136*** 0.123*** 0.164*** 0.160*** 0.255*** 0.235*** 0.141*** 0.152*** (0.032) (0.034) (0.025) (0.026) (0.054) (0.054) (0.034) (0.034) Judge and skater reside in same region 0.041 0.025 0.016 0.021 (0.027) (0.018) (0.039) (0.017) Log distance of skater club and judge’s residence 0.014* 0.006 −0.011 −0.011 (0.008) (0.006) (0.013) (0.013) Programme Length −0.022 −0.022 −0.046** −0.046** (0.023) (0.024) (0.021) (0.021) Same sex judge and skater 0.024 0.024 0.022 0.022 −0.103* −0.102* 0.018 0.018 (0.029) (0.028) (0.026) (0.026) (0.053) (0.054) (0.032) (0.032) Junior skater 0.404*** 0.404*** (0.059) (0.059) Senior skater 0.932*** 0.933*** (0.027) (0.027) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 47,880 47,880 47,880 47,880 47,880 47,880 47,880 47,880 R-squared 0.826 0.826 0.848 0.848 0.406 0.406 0.779 0.779 Source: Authors’ calculations. Note: The dependent variable is the programme component mark awarded by each judge to each skater for each component. The data comprises 13 competitions that make up the 2011–12 qualifying stream in singles. The following events are included: CP Regional 2012, EGL Regional 2012, Eastern Sectional 2012, Midwestern Sectional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, Nationals 2012, Pacific Coast Sectional 2012, SA Regional 2012, SW Regional 2012, UGL Regional 2012. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Table 3A The effect of shared club affiliation on programme component marks: All levels of qualifying competitions (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.136*** 0.123*** 0.164*** 0.160*** 0.255*** 0.235*** 0.141*** 0.152*** (0.032) (0.034) (0.025) (0.026) (0.054) (0.054) (0.034) (0.034) Judge and skater reside in same region 0.041 0.025 0.016 0.021 (0.027) (0.018) (0.039) (0.017) Log distance of skater club and judge’s residence 0.014* 0.006 −0.011 −0.011 (0.008) (0.006) (0.013) (0.013) Programme Length −0.022 −0.022 −0.046** −0.046** (0.023) (0.024) (0.021) (0.021) Same sex judge and skater 0.024 0.024 0.022 0.022 −0.103* −0.102* 0.018 0.018 (0.029) (0.028) (0.026) (0.026) (0.053) (0.054) (0.032) (0.032) Junior skater 0.404*** 0.404*** (0.059) (0.059) Senior skater 0.932*** 0.933*** (0.027) (0.027) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 47,880 47,880 47,880 47,880 47,880 47,880 47,880 47,880 R-squared 0.826 0.826 0.848 0.848 0.406 0.406 0.779 0.779 (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.136*** 0.123*** 0.164*** 0.160*** 0.255*** 0.235*** 0.141*** 0.152*** (0.032) (0.034) (0.025) (0.026) (0.054) (0.054) (0.034) (0.034) Judge and skater reside in same region 0.041 0.025 0.016 0.021 (0.027) (0.018) (0.039) (0.017) Log distance of skater club and judge’s residence 0.014* 0.006 −0.011 −0.011 (0.008) (0.006) (0.013) (0.013) Programme Length −0.022 −0.022 −0.046** −0.046** (0.023) (0.024) (0.021) (0.021) Same sex judge and skater 0.024 0.024 0.022 0.022 −0.103* −0.102* 0.018 0.018 (0.029) (0.028) (0.026) (0.026) (0.053) (0.054) (0.032) (0.032) Junior skater 0.404*** 0.404*** (0.059) (0.059) Senior skater 0.932*** 0.933*** (0.027) (0.027) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 47,880 47,880 47,880 47,880 47,880 47,880 47,880 47,880 R-squared 0.826 0.826 0.848 0.848 0.406 0.406 0.779 0.779 Source: Authors’ calculations. Note: The dependent variable is the programme component mark awarded by each judge to each skater for each component. The data comprises 13 competitions that make up the 2011–12 qualifying stream in singles. The following events are included: CP Regional 2012, EGL Regional 2012, Eastern Sectional 2012, Midwestern Sectional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, Nationals 2012, Pacific Coast Sectional 2012, SA Regional 2012, SW Regional 2012, UGL Regional 2012. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Table 3B Summary statistics for Table 3A Variable N Mean Std. Dev. Min Max Component mark assigned by judge 47,880 4.363 1.188 0.75 9.75 Judge and skater share club affiliation 47,880 0.057 0.231 0 1 Judge and skater reside in same region 47,880 0.458 .498 0 1 Log distance between skater club and judge residence 47,880 5.551 1.526 .0001 8.121 Programme Length 47,880 0.498 0.498 0 1 Junior level skater 47,880 0.318 0.460 0 1 Senior level skater 47,880 0.305 0.461 0 1 Same Sex Judge and Skater 47,880 0.645 0.479 0 1 Variable N Mean Std. Dev. Min Max Component mark assigned by judge 47,880 4.363 1.188 0.75 9.75 Judge and skater share club affiliation 47,880 0.057 0.231 0 1 Judge and skater reside in same region 47,880 0.458 .498 0 1 Log distance between skater club and judge residence 47,880 5.551 1.526 .0001 8.121 Programme Length 47,880 0.498 0.498 0 1 Junior level skater 47,880 0.318 0.460 0 1 Senior level skater 47,880 0.305 0.461 0 1 Same Sex Judge and Skater 47,880 0.645 0.479 0 1 Source: Authors’ calculations. Note: The data comprises 13 competitions that make up the 2011–12 qualifying stream in singles. The following events are included: CP Regional 2012, EGL Regional 2012, Eastern Sectional 2012, Midwestern Sectional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, Nationals 2012, Pacific Coast Sectional 2012, SA Regional 2012, SW Regional 2012 and UGL Regional 2012. Table 3B Summary statistics for Table 3A Variable N Mean Std. Dev. Min Max Component mark assigned by judge 47,880 4.363 1.188 0.75 9.75 Judge and skater share club affiliation 47,880 0.057 0.231 0 1 Judge and skater reside in same region 47,880 0.458 .498 0 1 Log distance between skater club and judge residence 47,880 5.551 1.526 .0001 8.121 Programme Length 47,880 0.498 0.498 0 1 Junior level skater 47,880 0.318 0.460 0 1 Senior level skater 47,880 0.305 0.461 0 1 Same Sex Judge and Skater 47,880 0.645 0.479 0 1 Variable N Mean Std. Dev. Min Max Component mark assigned by judge 47,880 4.363 1.188 0.75 9.75 Judge and skater share club affiliation 47,880 0.057 0.231 0 1 Judge and skater reside in same region 47,880 0.458 .498 0 1 Log distance between skater club and judge residence 47,880 5.551 1.526 .0001 8.121 Programme Length 47,880 0.498 0.498 0 1 Junior level skater 47,880 0.318 0.460 0 1 Senior level skater 47,880 0.305 0.461 0 1 Same Sex Judge and Skater 47,880 0.645 0.479 0 1 Source: Authors’ calculations. Note: The data comprises 13 competitions that make up the 2011–12 qualifying stream in singles. The following events are included: CP Regional 2012, EGL Regional 2012, Eastern Sectional 2012, Midwestern Sectional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, Nationals 2012, Pacific Coast Sectional 2012, SA Regional 2012, SW Regional 2012 and UGL Regional 2012. In all eight columns of Table 3A, the point estimates on skaters and judges sharing the same club affiliation are positive and statistically significant. The estimates show that when a judge and a skater share the same club affiliation, the skater’s mark increases between 0.14 and 0.26 points. Thus, our estimates show that the shared club affiliation leads, on average, to more than half of a one-increment increase in the mark awarded. The regressions in columns 1 and 2 include an indicator for performance to control for the quality of each performance. These first two columns differ with respect to our measure for the location variable. Columns 3 and 4 use identical explanatory variables as the first two columns but add fixed effects for each judge. Columns 1 and 3 include a variable for regional affiliation, identifying whether a judge resides in the same region as the skater's club. Columns 2 and 4 include log distance, measured as the natural logarithm of the number of miles between the residence of the judge and the location of the skater’s club. Columns 5 and 6 include indicators for each competition but do not include skater or judge fixed effects and include indicators for junior and senior levels. Columns 7 and 8 include similar explanatory variables as columns 5 and 6 but include skater and judge fixed effects along with competition fixed effects and drop the dummy variables for junior and senior since these variables are perfectly collinear with the skater fixed effects, given that a skater only competes in one level. We also include an indicator for whether a mark is for a free skating programme in the latter four specifications without performance fixed effects. The point estimates show that having the same club affiliation is associated with an increase in that mark of approximately 15% of the mark’s standard deviation. A conservative estimate is that sharing a club affiliation increases each component mark given by a judge by 2.5%. In a review of the results of qualifying competitions used in this study, almost 15% of event placements are within .25 points of each other. At the 2008 National Championships, Johnny Weir and Evan Lysacek tied for the men’s title. Both skaters earned a total of 244.77 points and a non-points tiebreaker was used. A previous study of Olympic diving by Emerson, Seltzer and Lin (2009) found nationalistic bias in diving results from the 2000 Summer Olympic Games. This study identified specific judges and divers who marked strategically higher or lower than an ‘unbiased dive score’ and found that the removal of judging bias might have changed the medal standings. Although our analysis does not include a similar ‘unbiased’ ranking, the small point differences in skating, between winners and runners-up suggests that bias could impact final results. The most recent study of figure skating (Zitzewitz, 2014) relies on data from international competition which lacks the numerical richness and transparency of the US domestic competitions and thus relied on the max-median and inter-quartile spreads in marks to measure nationalistic bias. Using this method, Zitzewitz found that having a compatriot on panel raised the sum of GOE and components scores by .39 points compared to our finding of an increase in just the components of .15 to .4 points (depending on programme length). Although each judge’s mark contributes toward the average mark, as each judge awards five marks to each skater, the cumulative effect is to increase a skater’s final score by the full amount of the estimated effect. Additionally, the components marks are factored into the total score with .8 of the total used for the short programme but with a factor of 1.6 for a free skate programme. Therefore, the increase in mark is amplified for the free skate programme. In Table 3A, columns 5 and 6 we also find that relative to novices, the average junior skater’s mark is 0.4 points higher and a senior skater’s mark is 0.93 points higher than the average score of novice skaters. This is to be expected as the skill level increases as a skater moves from novice to the higher levels. In light of the results in Table 2, one might wonder whether the results on the shared club affiliation variable are driven by the fact that judges who share a club affiliation with a skater tend to award the highest marks to that skater. To study this issue, we discard for each skater the highest mark for each component. If most of the biased marks are driven by the highest marks, then the impact of bias will be mitigated when we drop the high mark for each component for each programme and use the remaining marks to estimate the same regression specifications as we do in Table 3A. Table 4 shows that the point estimate on shared club affiliation remains positive and statistically significant. We dropped the highest marks, resulting in a new mean of the component marks assigned by a judge of 4.233. Here, a shared club affiliation leads to an increase in the judge’s mark by between 0.09-0.2, these point estimates are of slightly smaller than those in Table 3A. However, the sum of these results shows that the findings in Table 3A are not driven by only those judges who share a club affiliation with the skater and award the highest mark.7 Additional specifications, not reported in this work, which include the difference between one judge’s score and the mean of the other scores on the panel for the same performance, referred to as difference-based analysis (Popović, 2000; Emerson et al., 2009) provide the similar results of statistically significant bias in favour of same club skaters. Table 4 The effect of shared club affiliation on programme component marks: all levels of competition—highest marks excluded (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.094*** 0.087*** 0.127*** 0.126*** 0.199*** 0.186*** 0.127*** 0.141*** (0.030) (0.031) (0.020) (0.021) (0.056) (0.057) (0.031) (0.030) Judge and skater reside in same region 0.042** 0.029** 0.012 0.020 (0.021) (0.014) (0.037) (0.017) Log distance between skater and judge’s residence −0.011 −0.005 −0.008 0.005 (0.007) (0.004) (0.013) (0.007) Programme Length −0.022 −0.023 −0.034* −0.033* (0.025) (0.025) (0.020) (0.020) Same sex judge and skater 0.015 0.014 0.021 0.022 −0.107** −0.107** 0.025 0.024 (0.023) (0.023) (0.021) (0.021) (0.053) (0.053) (0.029) (0.029) Junior skater 0.394*** 0.394*** 0.394*** 0.394*** (0.060) (0.060) (0.060) (0.060) Senior skater 0.925*** 0.925*** 0.925*** 0.925*** (0.099) (0.099) (0.099) (0.099) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 37,146 37,146 37,146 37,146 37,146 37,146 37,146 37,146 R-squared 0.873 0.873 0.887 0.887 0.439 0.439 0.816 0.816 (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.094*** 0.087*** 0.127*** 0.126*** 0.199*** 0.186*** 0.127*** 0.141*** (0.030) (0.031) (0.020) (0.021) (0.056) (0.057) (0.031) (0.030) Judge and skater reside in same region 0.042** 0.029** 0.012 0.020 (0.021) (0.014) (0.037) (0.017) Log distance between skater and judge’s residence −0.011 −0.005 −0.008 0.005 (0.007) (0.004) (0.013) (0.007) Programme Length −0.022 −0.023 −0.034* −0.033* (0.025) (0.025) (0.020) (0.020) Same sex judge and skater 0.015 0.014 0.021 0.022 −0.107** −0.107** 0.025 0.024 (0.023) (0.023) (0.021) (0.021) (0.053) (0.053) (0.029) (0.029) Junior skater 0.394*** 0.394*** 0.394*** 0.394*** (0.060) (0.060) (0.060) (0.060) Senior skater 0.925*** 0.925*** 0.925*** 0.925*** (0.099) (0.099) (0.099) (0.099) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 37,146 37,146 37,146 37,146 37,146 37,146 37,146 37,146 R-squared 0.873 0.873 0.887 0.887 0.439 0.439 0.816 0.816 Source: Authors’ calculations. Note: The dependent variable is the programme component mark awarded by each judge to each skater for each component. The data comprises 13 competitions that make up the 2011–12 qualifying stream in singles. The following events are included: CP Regional 2012, EGL Regional 2012, Eastern Sectional 2012, Midwestern Sectional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, Nationals 2012, Pacific Coast Sectional 2012, SA Regional 2012, SW Regional 2012, UGL Regional 2012. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Table 4 The effect of shared club affiliation on programme component marks: all levels of competition—highest marks excluded (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.094*** 0.087*** 0.127*** 0.126*** 0.199*** 0.186*** 0.127*** 0.141*** (0.030) (0.031) (0.020) (0.021) (0.056) (0.057) (0.031) (0.030) Judge and skater reside in same region 0.042** 0.029** 0.012 0.020 (0.021) (0.014) (0.037) (0.017) Log distance between skater and judge’s residence −0.011 −0.005 −0.008 0.005 (0.007) (0.004) (0.013) (0.007) Programme Length −0.022 −0.023 −0.034* −0.033* (0.025) (0.025) (0.020) (0.020) Same sex judge and skater 0.015 0.014 0.021 0.022 −0.107** −0.107** 0.025 0.024 (0.023) (0.023) (0.021) (0.021) (0.053) (0.053) (0.029) (0.029) Junior skater 0.394*** 0.394*** 0.394*** 0.394*** (0.060) (0.060) (0.060) (0.060) Senior skater 0.925*** 0.925*** 0.925*** 0.925*** (0.099) (0.099) (0.099) (0.099) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 37,146 37,146 37,146 37,146 37,146 37,146 37,146 37,146 R-squared 0.873 0.873 0.887 0.887 0.439 0.439 0.816 0.816 (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.094*** 0.087*** 0.127*** 0.126*** 0.199*** 0.186*** 0.127*** 0.141*** (0.030) (0.031) (0.020) (0.021) (0.056) (0.057) (0.031) (0.030) Judge and skater reside in same region 0.042** 0.029** 0.012 0.020 (0.021) (0.014) (0.037) (0.017) Log distance between skater and judge’s residence −0.011 −0.005 −0.008 0.005 (0.007) (0.004) (0.013) (0.007) Programme Length −0.022 −0.023 −0.034* −0.033* (0.025) (0.025) (0.020) (0.020) Same sex judge and skater 0.015 0.014 0.021 0.022 −0.107** −0.107** 0.025 0.024 (0.023) (0.023) (0.021) (0.021) (0.053) (0.053) (0.029) (0.029) Junior skater 0.394*** 0.394*** 0.394*** 0.394*** (0.060) (0.060) (0.060) (0.060) Senior skater 0.925*** 0.925*** 0.925*** 0.925*** (0.099) (0.099) (0.099) (0.099) Performance Fixed Effect? Yes Yes Yes Yes No No No No Skater Fixed Effect? No No No No No No Yes Yes Judge Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No No No Yes Yes Yes Yes Observations 37,146 37,146 37,146 37,146 37,146 37,146 37,146 37,146 R-squared 0.873 0.873 0.887 0.887 0.439 0.439 0.816 0.816 Source: Authors’ calculations. Note: The dependent variable is the programme component mark awarded by each judge to each skater for each component. The data comprises 13 competitions that make up the 2011–12 qualifying stream in singles. The following events are included: CP Regional 2012, EGL Regional 2012, Eastern Sectional 2012, Midwestern Sectional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, Nationals 2012, Pacific Coast Sectional 2012, SA Regional 2012, SW Regional 2012, UGL Regional 2012. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. 5.2 Evaluation at the discontinuity in payoffs After the free skating programme and short programme scores are summed. The top four men and women, at each skill level advance from the regional competition to the sectional and from the sectional competition to the National Championships, while the season is over for those who finish fifth or lower. After the short programme, which is skated first, some skaters are more secure in advancing based on a strong performance while others, who are ranked very low, have a poor outlook. The skaters in fourth and fifth places after the short programme are on the cusp of advancement but are not guaranteed to advance until performance of the free skating programme and the computation of a final placement. For judges, the benefits of biased marking for these skaters on the bubble are higher in this case than the benefits of biased markings for low-ranked skaters. Therefore, we predict greater bias for skaters on the bubble. To test whether the bubble status affects the marking of the skater we include free skating programme observations for those skaters who were ranked fourth or fifth after the short programme. We include three indicators for the interaction between relationship and placement after the short programme. We do not include an indicator for placement position only because that indicator is perfectly collinear with performance and skater fixed effects. The first three columns of Table 58 show results for regional and sectional competitions. Table 5, column 1 shows approximately a .02-point increase in marks for skaters in fourth or fifth place ‘on the bubble’ and who share the same club affiliation with one of the judges. The point estimate is statistically significant at the 5% level. The positive point estimate implies that when a judge shares the same club membership as the skater, and the skater is on the bubble, this judge increases their mark. The remaining columns in Table 5 show that this result is robust to alternative regression specifications. These findings are consistent with the hypothesis that the largest bias occurs in situations where the bias is most likely to be a determining factor as to whether the skater will advance to the next level of competition. The point estimates for skaters in places 1–3 or 6–8 are not statistically significant, and we do not find evidence of bias for these groups of skaters when a judge shares a same club relationship with the skater. However, in part due to relatively large standard errors estimated for the skaters in places 1–3 and 6–8, an F-test does not allow us to reject that the coefficients for skaters placed 1–3 vs 5–6 are statistically different from each other. Nor can we reject the null hypothesis that skaters placed 6–8 vs 4–5 are statistically different from each other. In both tests, the p-value associated with the F-test is 0.11. Table 5 Strategic marking behaviour. Examination of skaters’ free skating scores after placement in short programme Regional and Sectional Competitions National National (1) (2) (3) (4) (5) Judge and skater share club affiliation 0.128*** 0.055 0.056 0.131* 0.132* (0.045) (0.063) (0.063) (0.076) (0.076) Short Programme Placement 1–3 interaction with shared club 0.003 0.120 0.119 0.147 0.149* (0.067) (0.096) (0.097) (0.099) (0.079) Short Programme Placement 4–5 interaction with shared club 0.022** 0.191** 0.191** 0.081 0.079 (0.011) (0.093) (0.093) (0.261) (0.265) Short Programme Placement 6–8 interaction with shared club −0.103 −0.031 −0.033 −0.012 0.040 (0.078) (0.106) (0.107) (0.000) (0.000) Judge and skater reside in same region 0.037 0.019 0.019 0.048 0.048 (0.025) (0.025) (0.024) (0.062) (0.062) Same sex judge and skater 0.003 −0.016 0.139*** 0.138*** (0.035) (0.050) (0.053) (0.052) Performance Fixed Effect? Yes No No Yes No Skater Fixed Effect? No Yes Yes No Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Competition Fixed Effect? No Yes Yes No Yes Observations 19,945 19,945 19,945 3,915 3,915 R-squared 0.734 0.734 0.734 0.877 0.877 Regional and Sectional Competitions National National (1) (2) (3) (4) (5) Judge and skater share club affiliation 0.128*** 0.055 0.056 0.131* 0.132* (0.045) (0.063) (0.063) (0.076) (0.076) Short Programme Placement 1–3 interaction with shared club 0.003 0.120 0.119 0.147 0.149* (0.067) (0.096) (0.097) (0.099) (0.079) Short Programme Placement 4–5 interaction with shared club 0.022** 0.191** 0.191** 0.081 0.079 (0.011) (0.093) (0.093) (0.261) (0.265) Short Programme Placement 6–8 interaction with shared club −0.103 −0.031 −0.033 −0.012 0.040 (0.078) (0.106) (0.107) (0.000) (0.000) Judge and skater reside in same region 0.037 0.019 0.019 0.048 0.048 (0.025) (0.025) (0.024) (0.062) (0.062) Same sex judge and skater 0.003 −0.016 0.139*** 0.138*** (0.035) (0.050) (0.053) (0.052) Performance Fixed Effect? Yes No No Yes No Skater Fixed Effect? No Yes Yes No Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Competition Fixed Effect? No Yes Yes No Yes Observations 19,945 19,945 19,945 3,915 3,915 R-squared 0.734 0.734 0.734 0.877 0.877 Source: Authors’ calculations. Note: Observations include free skating (long) programmes for the nine regional and three sectional competitions in columns 1–3. Dependent variable is the mark a judge assigns for a programme component. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Table 5 Strategic marking behaviour. Examination of skaters’ free skating scores after placement in short programme Regional and Sectional Competitions National National (1) (2) (3) (4) (5) Judge and skater share club affiliation 0.128*** 0.055 0.056 0.131* 0.132* (0.045) (0.063) (0.063) (0.076) (0.076) Short Programme Placement 1–3 interaction with shared club 0.003 0.120 0.119 0.147 0.149* (0.067) (0.096) (0.097) (0.099) (0.079) Short Programme Placement 4–5 interaction with shared club 0.022** 0.191** 0.191** 0.081 0.079 (0.011) (0.093) (0.093) (0.261) (0.265) Short Programme Placement 6–8 interaction with shared club −0.103 −0.031 −0.033 −0.012 0.040 (0.078) (0.106) (0.107) (0.000) (0.000) Judge and skater reside in same region 0.037 0.019 0.019 0.048 0.048 (0.025) (0.025) (0.024) (0.062) (0.062) Same sex judge and skater 0.003 −0.016 0.139*** 0.138*** (0.035) (0.050) (0.053) (0.052) Performance Fixed Effect? Yes No No Yes No Skater Fixed Effect? No Yes Yes No Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Competition Fixed Effect? No Yes Yes No Yes Observations 19,945 19,945 19,945 3,915 3,915 R-squared 0.734 0.734 0.734 0.877 0.877 Regional and Sectional Competitions National National (1) (2) (3) (4) (5) Judge and skater share club affiliation 0.128*** 0.055 0.056 0.131* 0.132* (0.045) (0.063) (0.063) (0.076) (0.076) Short Programme Placement 1–3 interaction with shared club 0.003 0.120 0.119 0.147 0.149* (0.067) (0.096) (0.097) (0.099) (0.079) Short Programme Placement 4–5 interaction with shared club 0.022** 0.191** 0.191** 0.081 0.079 (0.011) (0.093) (0.093) (0.261) (0.265) Short Programme Placement 6–8 interaction with shared club −0.103 −0.031 −0.033 −0.012 0.040 (0.078) (0.106) (0.107) (0.000) (0.000) Judge and skater reside in same region 0.037 0.019 0.019 0.048 0.048 (0.025) (0.025) (0.024) (0.062) (0.062) Same sex judge and skater 0.003 −0.016 0.139*** 0.138*** (0.035) (0.050) (0.053) (0.052) Performance Fixed Effect? Yes No No Yes No Skater Fixed Effect? No Yes Yes No Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Competition Fixed Effect? No Yes Yes No Yes Observations 19,945 19,945 19,945 3,915 3,915 R-squared 0.734 0.734 0.734 0.877 0.877 Source: Authors’ calculations. Note: Observations include free skating (long) programmes for the nine regional and three sectional competitions in columns 1–3. Dependent variable is the mark a judge assigns for a programme component. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Columns 4 and 5 of Table 5 shows results for national competitions, where we do not predict an effect on the interaction variable, since no skater automatically advances to international competitions. Consistent with this prediction, for the National Championship, we do not find a statistically significant effect on the interaction variable between the fourth or fifth placement and shared club variables. We find an increase in marks awarded when the skater and judge share a club affiliation and when skater and judge have a shared sex. However, since skaters only compete against skaters of the same sex, this would not be significant in determining final placements. 5.3 Effect of level of competition on affiliation bias To examine whether biased judging is occurring only at some types of competitions, in Table 6 we estimate separate regressions for each competition level for the 2011–12 singles event with the programme component mark as the dependent variable. All specifications include either performance and judge fixed effects that control for heterogeneity of judging and skating ability or competition, skater and judge fixed effects as an alternative method of controls. Further, as in the previous regressions, we cluster standard errors by both judge and competitor. Table 6 The effect of shared club affiliation on programme component marks: all levels of qualifying competitions, singles only, 2011–12 Regional Sectional National (same as table 7) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.190*** 0.172*** 0.172*** 0.100* 0.104* 0.101* 0.127** 0.207*** 0.110* (0.028) (0.039) (0.039) (0.054) (0.054) (0.057) (0.054) (0.039) (0.060) Judge and skater reside in same region 0.208*** 0.222*** 0.017 0.004 0.117*** 0.122*** (0.063) (0.065) (0.018) (0.018) (0.037) (0.039) Long Programme −0.055** −0.057** −0.069* −0.069* 0.013 0.013 (0.026) (0.026) (0.040) (0.040) (0.065) (0.065) Same sex judge and skater −0.047 0.004 0.133*** (0.040) (0.039) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 27,665 27,665 27,665 12,385 12,385 12,385 7,830 7,830 7,830 R-squared 0.777 0.704 0.704 0.771 0.707 0.707 0.848 0.813 0.813 Regional Sectional National (same as table 7) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.190*** 0.172*** 0.172*** 0.100* 0.104* 0.101* 0.127** 0.207*** 0.110* (0.028) (0.039) (0.039) (0.054) (0.054) (0.057) (0.054) (0.039) (0.060) Judge and skater reside in same region 0.208*** 0.222*** 0.017 0.004 0.117*** 0.122*** (0.063) (0.065) (0.018) (0.018) (0.037) (0.039) Long Programme −0.055** −0.057** −0.069* −0.069* 0.013 0.013 (0.026) (0.026) (0.040) (0.040) (0.065) (0.065) Same sex judge and skater −0.047 0.004 0.133*** (0.040) (0.039) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 27,665 27,665 27,665 12,385 12,385 12,385 7,830 7,830 7,830 R-squared 0.777 0.704 0.704 0.771 0.707 0.707 0.848 0.813 0.813 Source: Authors’ calculations. Note: Dependent variable is the mark a judge assigns for a programme component. The data comprises 13 competitions. The regional competitions included are the CP Regional 2012, EGL Regional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, SA Regional 2012, SW Regional 2012, SWP Regional 2012, and UGL Regional 2012. The sectional competitions are the Eastern Sectional 2012, Midwestern Sectional 2012, Pacific Coast Sectional 2012. The last three columns are based on the National Competition 2012. Standard errors are clustered by competitor and judge. *** p < 0.01, ** p < 0.05, * p < 0.1, two-tailed test. Table 6 The effect of shared club affiliation on programme component marks: all levels of qualifying competitions, singles only, 2011–12 Regional Sectional National (same as table 7) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.190*** 0.172*** 0.172*** 0.100* 0.104* 0.101* 0.127** 0.207*** 0.110* (0.028) (0.039) (0.039) (0.054) (0.054) (0.057) (0.054) (0.039) (0.060) Judge and skater reside in same region 0.208*** 0.222*** 0.017 0.004 0.117*** 0.122*** (0.063) (0.065) (0.018) (0.018) (0.037) (0.039) Long Programme −0.055** −0.057** −0.069* −0.069* 0.013 0.013 (0.026) (0.026) (0.040) (0.040) (0.065) (0.065) Same sex judge and skater −0.047 0.004 0.133*** (0.040) (0.039) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 27,665 27,665 27,665 12,385 12,385 12,385 7,830 7,830 7,830 R-squared 0.777 0.704 0.704 0.771 0.707 0.707 0.848 0.813 0.813 Regional Sectional National (same as table 7) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.190*** 0.172*** 0.172*** 0.100* 0.104* 0.101* 0.127** 0.207*** 0.110* (0.028) (0.039) (0.039) (0.054) (0.054) (0.057) (0.054) (0.039) (0.060) Judge and skater reside in same region 0.208*** 0.222*** 0.017 0.004 0.117*** 0.122*** (0.063) (0.065) (0.018) (0.018) (0.037) (0.039) Long Programme −0.055** −0.057** −0.069* −0.069* 0.013 0.013 (0.026) (0.026) (0.040) (0.040) (0.065) (0.065) Same sex judge and skater −0.047 0.004 0.133*** (0.040) (0.039) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 27,665 27,665 27,665 12,385 12,385 12,385 7,830 7,830 7,830 R-squared 0.777 0.704 0.704 0.771 0.707 0.707 0.848 0.813 0.813 Source: Authors’ calculations. Note: Dependent variable is the mark a judge assigns for a programme component. The data comprises 13 competitions. The regional competitions included are the CP Regional 2012, EGL Regional 2012, NA Regional 2012, NE Regional 2012, NWP Regional 2012, SA Regional 2012, SW Regional 2012, SWP Regional 2012, and UGL Regional 2012. The sectional competitions are the Eastern Sectional 2012, Midwestern Sectional 2012, Pacific Coast Sectional 2012. The last three columns are based on the National Competition 2012. Standard errors are clustered by competitor and judge. *** p < 0.01, ** p < 0.05, * p < 0.1, two-tailed test. The weaker skaters are eliminated through the qualifying process and the average skating quality increases as the season progresses from regional to sectional to national competition. Meanwhile, while national, sectional and regional judges serve as officials and award marks at regional events, only national judges participate in the national event and only sectional and national judges participate in the sectional event. We find a positive and statistically significant coefficient on same-club affiliation for all levels of competition, showing that marks increase when the skater and the judge share the same club affiliation. The point estimates for the regional, sectional and national competitions are between 0.10 and 0.21. Judges share an affiliation for 6.6% of the marks in the regional competitions, 2.1% at sectional competitions, and 2.7% at the national competitions. In many of the specifications, judges who reside in the same region as a skater increase the mark awarded on top of the shared club affiliation. Because shared region is almost always concurrent with a shared club affiliation this may compound the escalation of marks. There is some variation in the point estimates of the club affiliation variable for different levels of competition, this finding could either be due to a more heterogeneous judging pool in lower level competitions, or due to the fact that at higher level competitions, judges have a stronger incentive to mark accurately, due to the increased publicity associated with higher level competitions. To distinguish between these two explanations, in Table 7 we re-estimate the specification of six for national judges.9 Only national judges can serve at regional, sectional and national competitions. We drop any marks from the regional and sectional competitions in which there were fewer than three national judges on the official panel. We find that national judges display bias from shared club affiliation in all levels of competition. For all competitions, the 95% confidence intervals of the estimates on same-club affiliation overlap. Table 7 The effect of shared club affiliation on programme component marks. NATIONAL JUDGES ONLY—all levels of qualifying competitions, singles only, 2011–12 Regional Sectional National (same as Table 6) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.121*** 0.145*** 0.145*** 0.099* 0.114** 0.108* 0.127** 0.207*** 0.110* (0.037) (0.056) (0.056) (0.059) (0.057) (0.060) (0.054) (0.039) (0.060) Judge and skater reside in same Region 0.309** 0.057 0.015 0.008 0.117*** 0.122*** (0.156) (0.072) (0.022) (0.022) (0.037) (0.039) Long Programme −0.057 −0.057 −0.082* −0.082* 0.013 0.013 (0.038) (0.039) (0.043) (0.043) (0.065) (0.065) Same sex judge and skater −0.057 0.023 0.133*** (0.063) (0.041) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 12,190 12,190 12.190 9,415 9,415 9,415 7,830 7,830 7,830 R-squared 0.817 0.736 0.736 0.776 0.707 0.707 0.848 0.813 0.813 Regional Sectional National (same as Table 6) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.121*** 0.145*** 0.145*** 0.099* 0.114** 0.108* 0.127** 0.207*** 0.110* (0.037) (0.056) (0.056) (0.059) (0.057) (0.060) (0.054) (0.039) (0.060) Judge and skater reside in same Region 0.309** 0.057 0.015 0.008 0.117*** 0.122*** (0.156) (0.072) (0.022) (0.022) (0.037) (0.039) Long Programme −0.057 −0.057 −0.082* −0.082* 0.013 0.013 (0.038) (0.039) (0.043) (0.043) (0.065) (0.065) Same sex judge and skater −0.057 0.023 0.133*** (0.063) (0.041) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 12,190 12,190 12.190 9,415 9,415 9,415 7,830 7,830 7,830 R-squared 0.817 0.736 0.736 0.776 0.707 0.707 0.848 0.813 0.813 Source: Authors’ calculations. Note: Dependent variable is the mark a judge assigns for a programme component. The data comprises 13 competitions. The last three columns are based on the National Championships 2012 and are identical to columns 7–9 in Table 6. For columns 1–6, panels consisting of less than three national level judges at the regional and sectional competitions have been dropped. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Table 7 The effect of shared club affiliation on programme component marks. NATIONAL JUDGES ONLY—all levels of qualifying competitions, singles only, 2011–12 Regional Sectional National (same as Table 6) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.121*** 0.145*** 0.145*** 0.099* 0.114** 0.108* 0.127** 0.207*** 0.110* (0.037) (0.056) (0.056) (0.059) (0.057) (0.060) (0.054) (0.039) (0.060) Judge and skater reside in same Region 0.309** 0.057 0.015 0.008 0.117*** 0.122*** (0.156) (0.072) (0.022) (0.022) (0.037) (0.039) Long Programme −0.057 −0.057 −0.082* −0.082* 0.013 0.013 (0.038) (0.039) (0.043) (0.043) (0.065) (0.065) Same sex judge and skater −0.057 0.023 0.133*** (0.063) (0.041) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 12,190 12,190 12.190 9,415 9,415 9,415 7,830 7,830 7,830 R-squared 0.817 0.736 0.736 0.776 0.707 0.707 0.848 0.813 0.813 Regional Sectional National (same as Table 6) (1) (2) (3) (4) (5) (6) (7) (8) (9) Judge and skater share club affiliation 0.121*** 0.145*** 0.145*** 0.099* 0.114** 0.108* 0.127** 0.207*** 0.110* (0.037) (0.056) (0.056) (0.059) (0.057) (0.060) (0.054) (0.039) (0.060) Judge and skater reside in same Region 0.309** 0.057 0.015 0.008 0.117*** 0.122*** (0.156) (0.072) (0.022) (0.022) (0.037) (0.039) Long Programme −0.057 −0.057 −0.082* −0.082* 0.013 0.013 (0.038) (0.039) (0.043) (0.043) (0.065) (0.065) Same sex judge and skater −0.057 0.023 0.133*** (0.063) (0.041) (0.044) Performance Fixed Effect? Yes No No Yes No No Yes No No Skater Fixed Effect? No Yes Yes No Yes Yes No Yes Yes Competition Fixed Effect? No Yes Yes No Yes Yes N/A N/A N/A Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Yes Observations 12,190 12,190 12.190 9,415 9,415 9,415 7,830 7,830 7,830 R-squared 0.817 0.736 0.736 0.776 0.707 0.707 0.848 0.813 0.813 Source: Authors’ calculations. Note: Dependent variable is the mark a judge assigns for a programme component. The data comprises 13 competitions. The last three columns are based on the National Championships 2012 and are identical to columns 7–9 in Table 6. For columns 1–6, panels consisting of less than three national level judges at the regional and sectional competitions have been dropped. Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. 5.4 Grade of Execution vs component marks For the technical scores, each judge awards a GOE mark for each element in a programme. The number of elements depends on the skater or team skill level, and on whether it is a free skating or short programme. For each element, GOE marks range from -3 to + 3 in increments of 1. Our unit of analysis for the technical marking is the individual GOE mark given by a judge for a given skater for a single element in a programme. We test the hypothesis that the evaluation of programme component marks is more biased than the evaluation of GOE marks. Two institutional features motivate this hypothesis. First, with GOE marks judges have less discretion. The guidelines for awarding GOE marks have very specific directions for a base GOE mark with deductions and reductions for errors and increases for positive aspects. Second, for GOE marks, an increase of one increment is equivalent to a 0.728 standard deviation, while if the component mark is increased by one increment, the result is a 0.2 standard deviation increase. Because a GOE one increment increase is larger in terms of standard deviation, a one increment increase in a GOE mark is more likely to be noticed than an increment increase in component marks. The first four columns of Table 8 present the regression results for the GOE marks for singles competitions for the 2011–12 season.10 For the last four columns reproduce the analog specifications for component marks that we had presented in Table 3A. In each of the first four columns, we find that the point estimate on same club affiliation is less for the GOE marks than for the component marks. The estimates for the GOE marks range from 0.027 to 0.038 while the programme component marks range from 0.141 to 0.164. Thus, judges appear to be more objective and show less favouritism in awarding marks when they have less discretion. Table 8 Comparison of Grade of Execution marks vs component marks: all levels of qualifying competitions, singles only, 2011–12 Dependent Variable GOE Programme Component (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.038** 0.027* 0.037* 0.034* 0.164*** 0.160*** 0.141*** 0.152*** (0.015) (0.016) (0.013) (0.016) (0.025) (0.026) (0.034) (0.034) Judge and skater reside in same region 0.029** 0.031* 0.021 (0.012) (0.019) (0.018) (0.017) Log distance between skater club and judge’s residence −0.010** −0.006 0.006 −0.011 (0.004) (0.007) (0.006) (0.013) Long Programme 0.034 0.034 −0.046** −0.046** (0.028) (0.028) (0.021) (0.021) Same sex judge and skater −0.005 −0.004 0.022 0.022 (0.012) (0.012) (0.026) (0.026) Performance Fixed Effect? Yes Yes No No Yes Yes No No Skater Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No Yes Yes No No Yes Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Observations 84,309 84,309 84,309 84,309 47,880 47,880 47,880 47,880 R-squared 0.204 0.204 0.150 0.150 0.848 0.848 0.779 0.779 Dependent Variable GOE Programme Component (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.038** 0.027* 0.037* 0.034* 0.164*** 0.160*** 0.141*** 0.152*** (0.015) (0.016) (0.013) (0.016) (0.025) (0.026) (0.034) (0.034) Judge and skater reside in same region 0.029** 0.031* 0.021 (0.012) (0.019) (0.018) (0.017) Log distance between skater club and judge’s residence −0.010** −0.006 0.006 −0.011 (0.004) (0.007) (0.006) (0.013) Long Programme 0.034 0.034 −0.046** −0.046** (0.028) (0.028) (0.021) (0.021) Same sex judge and skater −0.005 −0.004 0.022 0.022 (0.012) (0.012) (0.026) (0.026) Performance Fixed Effect? Yes Yes No No Yes Yes No No Skater Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No Yes Yes No No Yes Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Observations 84,309 84,309 84,309 84,309 47,880 47,880 47,880 47,880 R-squared 0.204 0.204 0.150 0.150 0.848 0.848 0.779 0.779 Source: Authors’ calculations. Note: Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. Table 8 Comparison of Grade of Execution marks vs component marks: all levels of qualifying competitions, singles only, 2011–12 Dependent Variable GOE Programme Component (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.038** 0.027* 0.037* 0.034* 0.164*** 0.160*** 0.141*** 0.152*** (0.015) (0.016) (0.013) (0.016) (0.025) (0.026) (0.034) (0.034) Judge and skater reside in same region 0.029** 0.031* 0.021 (0.012) (0.019) (0.018) (0.017) Log distance between skater club and judge’s residence −0.010** −0.006 0.006 −0.011 (0.004) (0.007) (0.006) (0.013) Long Programme 0.034 0.034 −0.046** −0.046** (0.028) (0.028) (0.021) (0.021) Same sex judge and skater −0.005 −0.004 0.022 0.022 (0.012) (0.012) (0.026) (0.026) Performance Fixed Effect? Yes Yes No No Yes Yes No No Skater Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No Yes Yes No No Yes Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Observations 84,309 84,309 84,309 84,309 47,880 47,880 47,880 47,880 R-squared 0.204 0.204 0.150 0.150 0.848 0.848 0.779 0.779 Dependent Variable GOE Programme Component (1) (2) (3) (4) (5) (6) (7) (8) Judge and skater share club affiliation 0.038** 0.027* 0.037* 0.034* 0.164*** 0.160*** 0.141*** 0.152*** (0.015) (0.016) (0.013) (0.016) (0.025) (0.026) (0.034) (0.034) Judge and skater reside in same region 0.029** 0.031* 0.021 (0.012) (0.019) (0.018) (0.017) Log distance between skater club and judge’s residence −0.010** −0.006 0.006 −0.011 (0.004) (0.007) (0.006) (0.013) Long Programme 0.034 0.034 −0.046** −0.046** (0.028) (0.028) (0.021) (0.021) Same sex judge and skater −0.005 −0.004 0.022 0.022 (0.012) (0.012) (0.026) (0.026) Performance Fixed Effect? Yes Yes No No Yes Yes No No Skater Fixed Effect? No No Yes Yes No No Yes Yes Competition Fixed Effect? No No Yes Yes No No Yes Yes Judge Fixed Effect? Yes Yes Yes Yes Yes Yes Yes Yes Observations 84,309 84,309 84,309 84,309 47,880 47,880 47,880 47,880 R-squared 0.204 0.204 0.150 0.150 0.848 0.848 0.779 0.779 Source: Authors’ calculations. Note: Standard errors are clustered by competitor and judge. ***p < 0.01, **p < 0.05, *p < 0.1, two-tailed test. 6. Conclusion Using data from figure skating competitions in the USA, we study bias in judging decisions and find that judges show favouritism in awarding higher marks to skaters with whom they share the same club affiliation. This finding, that is, that shared group identity is a determinant of judging decisions, is consistent with theoretical work on identity and individual decision-making. We document that judges make biased decisions due to shared membership in the club with the skater, even when controlling for physical proximity as measured by the distance between judges’ homes and skaters’ clubs. Thus, as a social construct, a club often elicits greater affiliation bias than bias due to physical proximity. We also provide evidence that judges may act strategically. An additional finding is that decisions that allow for greater judging discretion are subject to larger bias than decisions subject to stricter guidelines. These results are relevant to many contexts when designing rule structures with the objective to limit the discretion of the evaluator so as to decrease the impact of bias. While group identity of a skating club is not as meaningful or salient as race, sex or nationality, our findings for this less salient attribute are still are consistent with bias. The impact of group membership on the evaluation of performance is relevant for the design of institutions for decision-making in hiring and promotion processes, in political and legal systems and other non-sports settings. However, as in many other settings, one of the difficulties in designing such institutions, as in figure skating, is the constraint that the pool of individuals that are qualified and willing to serve as evaluators is limited. For figure skating, the pool of judges consists of individuals who are deeply involved in the sport, are passionate about skating and who have longstanding personal relationships with others in the skating community. In corporate settings, businesses often attempt to broaden board rooms or employees by diversifying by race, gender and other, easily observable group identities. However, in contrast to diversifying with respect to the latter characteristics, it is much more complex to diversify with respect to type of group of identity that has a low salience, similar to skating club memberships. Our findings suggest that identifying and perhaps screening for less visible group memberships might be appropriate in settings in which there are incentives for individuals to show favouritism for in-group members. To date, figure skating rules do include conflict of interest policies for judges. In this sport, conflict of interest rules are solely limited to family members and those with financial relationships with skaters. Ceteris paribus, our results suggest that implementing of stricter conflict of interest policies, based on same club membership is predicted to reduce biased judging in figure skating. From personal interviews with judges, we learned that skating judges do not believe that they favour members of their club. Instead, they report that they tend to judge club members more harshly. One potential explanation to explain this apparent contradiction between our results and their beliefs, is that skating judges increase their marks to compensate for their belief that they judge skaters from their club too harshly. In this situation, additional information has the potential to reduce biased judging. Mere documentation of bias might raise judges’ awareness and can give them a sufficient incentive to change their behaviour. One example supporting this conjecture is that racial bias by officials in the National Basketball Association was reduced not through institutional changes but seemingly by making basketball referees aware of their own bias (Pope et al., 2013). The findings in this paper provide some support to the view that bias-reduction can be achieved through the selection of a pool of evaluators who are somewhat removed, in terms of group identity, from the person to be judged. One limitation of our data set is that we can only recognize and account for transparent relationships between skaters and judges. Over time, many individual skaters change locations to train. Some skaters retain membership in their initial club while others change membership. Judges, also, relocate and change club affiliations for work and family reasons. Because we can only identify the most transparent relationships, our empirical analysis may only identify the lower bound of any judging bias. This feature of our data resembles other settings in society in which objective evaluation is expected, such as from hiring committees and job applicants, or judges and attorneys, where potential shared group memberships may be largely invisible. Supplementary material The Appendix and Data files are available online at the OUP website. Footnotes 1 In 2016, the International Skating Union, with the support of the US Figure Skating Association changed the rules for international competition to mandate the identification of judges and marks awarded in international competitions including future Olympic and World Championship competitions. This rule change will allow for future studies on nationalism in figure skating judging. 2 For our sample period, juvenile and intermediate skaters do not compete in sectionals, but compete at a junior national event. 3 See http://usfigureskating.org/content/2014-15%20Rulebook%2008-14-14.pdf- (accessed 1 March 2015). 4 It is possible that a skater and judge shared a club affiliation prior to the competitions, we consider this, but no longer apply it to our data set, because the judge now belongs to a different club. We do not observe such past affiliations. To the extent that such past affiliations exist for some of our observations, unidentified relationships tend to increase favouritism, so we can view our estimates as a lower bound. 5 Given that there are five component marks, without any ties, only 20% of the marks would be the highest mark. However, because of ties there can be two highest values, and thus the percentage of high marks is greater than 20%. 6 The official ISU’s documentation explains that a component mark of 5 connotes average performance of the criteria, with a 4 being fair. This means that the average programme observed in the US qualifying stream is below average. 7 Summary statistics for Table 4 are presented in the Appendix Table A2. 8 Appendix Table A3 presents the summary statistics for Table 5. 9 We report the summary statistics for the variables of Table 6 in Appendix Table A4 while the variables of Table 7 can be found in Appendix Table A5. 10 Appendix Table A6 contains the summary statistics for Table 8 References 2011–12 Rulebook.pdf. Available at: http://usfsa.org/Content/201112Rulebook.pdf (accessed 1 March 2015). Adams M. ( 2011 ) Artistic Impressions: Figure Skating, Masculinity, and the Limits of Sport , University of Toronto Press , Toronto . Akerlof G. , Kranton R. ( 2000 ) Economics and identity , The Quarterly Journal of Economics , 115 , 715 – 53 . Google Scholar CrossRef Search ADS Bar T. , Zussman A. ( 2012 ) Partisan grading , American Economic Journal: Applied Economics , 4 , 30 – 48 . Google Scholar CrossRef Search ADS Bertrand M. , Chugh D. , Mullainathan S. ( 2005 ) Implicit discrimination , The American Economic Review , 95 , 94 – 8 . Google Scholar CrossRef Search ADS Charness G. , Rigotti L. , Rustichini A. ( 2007 ) Individual behavior and group membership, American Economic Review , 97 , 1340 – 52 . Google Scholar CrossRef Search ADS Chen Y. , Li S. ( 2009 ) Group identity and social preferences , The American Economic Review , 99 , 431 – 57 . Google Scholar CrossRef Search ADS Emerson J. , Arnold T. ( 2011 ) Statistical sleuthing by leveraging human nature: a study of Olympic figure skating, The American Statistician , 65 , 143 – 8 . Google Scholar CrossRef Search ADS Emerson J. , Meredith S. ( 2011 ) Nationalistic judging bias in the 2000 Olympic diving competition , Math Horizons , 18 , 8 – 11 . Google Scholar CrossRef Search ADS Emerson J. , Seltzer M. , Lin D. ( 2009 ) Assessing judging bias: an example from the 2000 Olympic games , The American Statistician , 63 , 124 – 31 . Google Scholar CrossRef Search ADS Goette L. , Huffman D. , Meier S. ( 2006 ) The impact of group membership on cooperation and norm enforcement: evidence using random assignment to real social groups , The American Economic Review , 96 , 212 – 6 . Google Scholar CrossRef Search ADS Kestnbaum E. ( 2003 ) Culture on Ice: Figure Skating and Cultural Meaning , Wesleyan University Press , Middletown, CT . Kim J. , King B. ( 2014 ) Seeing stars: matthew effects and status bias in major league baseball umpiring , Management Science , 60 , 2619 – 44 . Google Scholar CrossRef Search ADS Leskovsek B. , Cuk I. , Pajek J. , Forbes W. , Buvcar-Pajek M. ( 2012 ) Bias of judging in men’s artistic gymnastics at the European championship 2011 , Biology of Sport , 29 , 107 – 13 . Google Scholar CrossRef Search ADS Looney M. ( 2012 ) Judging anomalies at the 2010 Olympics in men’s figure skating , Measurement in Physical Education and Exercise Science , 16 , 55 – 68 . Google Scholar CrossRef Search ADS Parsons C. , Sulaeman J. , Yates M. , Hamermesh D. ( 2011 ) Strike three: discrimination, incentives, and evaluation, American Economic Review , 101 , 1410 – 35 . Google Scholar CrossRef Search ADS Pope D. , Price J. , Wolfers J. ( 2013 ) Awareness Reduces Racial Bias (Working Paper No. 19765), National Bureau of Economic Research, Cambridge, MA. Popović R. ( 2000 ) International bias detected in judging rhythmic gymnastics competition at Sydney 2000 Olympic Games , Facta Universitatis-Series: Physical Education and Sport , 1 , 1 – 13 . Price J. , Wolfers J. ( 2010 ) Racial discrimination among NBA referees, Quarterly Journal of Economics , 125 , 1859 – 87 . Google Scholar CrossRef Search ADS Shayo M. , Zussman A. ( 2011 ) Judicial ingroup bias in the shadow of terrorism , The Quarterly Journal of Economics , 126 , 1447 – 84 . Google Scholar CrossRef Search ADS Tajfel H. ( 1970 ) Experiments in intergroup discrimination, Scientific American , 223 , 96 – 102 . Google Scholar CrossRef Search ADS PubMed Whissell R. , Lyons S. , Wilkinson D. , Whissell C. ( 1993 ) National bias in judgements of Olympic-level skating, Perceptual and Motor Skills , 77 , 355 – 8 . Google Scholar CrossRef Search ADS Zitzewitz E. ( 2006 ) Nationalism in winter sports judging and its lessons for organizational decision making, Journal of Economics and Management Strategy , 15 , 67 – 99 . Google Scholar CrossRef Search ADS Zitzewitz E. ( 2014 ) Does transparency reduce favoritism and corruption? Evidence from the reform of figure skating judging , Journal of Sports Economics , 15 , 3 – 30 . Google Scholar CrossRef Search ADS © Oxford University Press 2018 All rights reserved This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

Oxford Economic PapersOxford University Press

Published: Jan 31, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off