tag:blogger.com,1999:blog-8391293127288856260.post7572912910818967838..comments2023-09-27T12:44:03.592+01:00Comments on MsHedgehog: Judging in the Mundial Final (Fun with Data)msHedgehoghttp://www.blogger.com/profile/05719152265628932122noreply@blogger.comBlogger10125tag:blogger.com,1999:blog-8391293127288856260.post-56241237293033549802016-07-28T23:02:34.243+01:002016-07-28T23:02:34.243+01:00@Yokoito: indeed. I agree with you that this is ar...@Yokoito: indeed. I agree with you that this is arguably a good thing, although I also think the Mundial has a somewhat paradoxical role in protecting tango from ballroomisation. Because all the finalists indisputably have good looking technique, whereas there are ballroom schools teaching a genuinely ballroomised argentine tango with a totally inappropriate (IMV) ballroom technique and approach. However, that's a subject for further research.<br /><br />@Mike: Judges rank a lot of couples equal with one another. They don't give forty different marks to forty different couples, and that really shows in the cross-plots of rankings. I haven't done the marks seperately from the rankings, but it implies that some judges at least give a restricted set of integers as marks, they don't try to make fine distinctions. They only see each couple dance once - have a look at the post on Music in the Mundial for a description of the procedure, and links to video.msHedgehoghttps://www.blogger.com/profile/05719152265628932122noreply@blogger.comtag:blogger.com,1999:blog-8391293127288856260.post-39909918754977550222016-07-28T14:46:32.830+01:002016-07-28T14:46:32.830+01:00Yes, having some slightly guilty fun - tangueros a...Yes, having some slightly guilty fun - tangueros are supposed to be in a trance rather than playing with stats, no?<br /><br />Anyway, I thought about how to visualise this. I'm not sure what you're planning but I took a cross plot of the rankings for each pair of judges. If they agreed perfectly the points would lie on the 45 degree line. However, as you have obviously found, they're nothing like this. I have a neat figure showing all cross plots for a single year in the same picture, but unfortunately cannot post this in a comment!<br /><br />I did wonder about the scores. Over how many dances are they averaged, for any given couple. Some judges (eg Garcia) only have averages that are integers or in steps of 0.5. Others (eg Brufman) have averages such as 7.05. Does this mean Brufman is finding fine distinctions between couples or is it the result of an averaging over several dances? Given this, rankings might be better.<br /><br />Hope you're enjoying this too! Have you tried graphing the top or bottom five only, or some such?<br /><br />Mike.<br /><br /><br />El escritorhttps://www.blogger.com/profile/10020567455325195391noreply@blogger.comtag:blogger.com,1999:blog-8391293127288856260.post-44997650350792971182016-07-28T13:29:32.022+01:002016-07-28T13:29:32.022+01:00Congratulations for all this number crunching...I ...Congratulations for all this number crunching...I am deeply impressed and also a bit jealous about the spare time you must have. In any case, I agree that it is a good thing that Tango is not (yet) as standardized and regulated as ballroom. And I hope it will never be. So maybe the scores given are just for "it looks good" which would explain the large spread and the non-correlations.Yokoitohttp://tangoblogblog.wordpress.comnoreply@blogger.comtag:blogger.com,1999:blog-8391293127288856260.post-49463205074829724362016-07-27T20:43:23.868+01:002016-07-27T20:43:23.868+01:00I'm so pleased that you're having such fun...I'm so pleased that you're having such fun.<br /><br />Thank you for the rank correlations; this is rather complicated to do in DAX for PowerBI, so it's super helpful to get an independent answer. I now get almost the same results for 2015, and I suspect the small differences are due to rounding at various stages of the calculation, so I will go ahead and pull in the other years. The scatter plots are pleasingly chaotic-looking, with intriguing flashes of seeming order.<br /><br />Looking at the visuals, I see very little indication that there's any agreement at any level. You'd expect the bottom five to be the most likely place to find agreement if there is any, partly because of the 'floor'. Of course, most people feel there's more agreement generally on bad dancing than good, but the kind of bad that people agree on should have been eliminated by the final.msHedgehoghttps://www.blogger.com/profile/05719152265628932122noreply@blogger.comtag:blogger.com,1999:blog-8391293127288856260.post-89165473069610990852016-07-27T10:05:44.003+01:002016-07-27T10:05:44.003+01:00... and the rank correlations:
Rank correlations
...... and the rank correlations:<br /><br />Rank correlations<br /><br />2015<br />(obs=41)<br /><br /> | blanco roldan delarosa torres copes cejas galera<br />-------------+---------------------------------------------------------------<br /> blanco | 1.0000 <br /> roldan | 0.2213 1.0000 <br /> delarosa | 0.1491 0.2063 1.0000 <br /> torres | 0.2315 0.2913 0.2966 1.0000 <br /> copes | 0.1032 0.2042 0.0159 -0.0777 1.0000 <br /> cejas | 0.1779 0.1947 0.3707 0.3181 0.1116 1.0000 <br /> galera | 0.0428 0.3602 0.1961 0.0925 0.3121 0.0208 1.0000 <br /><br /><br /><br />B[7,1]<br /> c1<br />r1 0.28<br />r2 0.35<br />r3 0.32<br />r4 0.31<br />r5 0.24<br />r6 0.31<br />r7 0.29<br /><br /><br />2014<br />(obs=41)<br /><br /> | eduard~i JorgeF~o Javier~z juliob~a VilmaV~a claude~a OlgaBe~o<br />-------------+---------------------------------------------------------------<br />eduardomasci | 1.0000 <br /> JorgeFirpo | -0.1276 1.0000 <br />JavierRodr~z | -0.0400 0.1785 1.0000 <br />juliobalma~a | -0.0800 0.2986 0.1484 1.0000 <br /> VilmaVega | 0.0296 0.1269 0.3057 0.1519 1.0000 <br />claudelina~a | -0.1837 0.1695 0.3939 0.3282 0.2335 1.0000 <br /> OlgaBesio | 0.1266 0.1164 -0.0595 0.1424 0.2080 0.0124 1.0000 <br /><br /><br /><br />B[7,1]<br /> c1<br />r1 0.10<br />r2 0.25<br />r3 0.28<br />r4 0.28<br />r5 0.29<br />r6 0.28<br />r7 0.22<br /><br /><br />2013<br />(obs=40)<br /><br /> | zotto martin~y ermocida schapira roldan brufman garcia<br />-------------+---------------------------------------------------------------<br /> zotto | 1.0000 <br /> martinezpey | 0.2939 1.0000 <br /> ermocida | 0.6370 0.4660 1.0000 <br /> schapira | 0.2271 0.2161 0.2158 1.0000 <br /> roldan | 0.2949 0.1630 0.2614 0.5297* 1.0000 <br /> brufman | 0.3879 0.1293 0.1812 0.2372 0.5121* 1.0000 <br /> garcia | -0.0170 0.1450 0.0858 0.1841 0.2390 0.1721 1.0000 <br /><br /><br /><br />B[7,1]<br /> c1<br />r1 0.40<br />r2 0.34<br />r3 0.41<br />r4 0.37<br />r5 0.43<br />r6 0.37<br />r7 0.26<br /><br /><br />2012<br />(obs=41)<br /><br /> | nieves besio balmac~a duplaa lubiz rodrig~z juarez<br />-------------+---------------------------------------------------------------<br /> nieves | 1.0000 <br /> besio | 0.2709 1.0000 <br /> balmaceda | 0.0935 0.1732 1.0000 <br /> duplaa | 0.2814 0.0504 0.0291 1.0000 <br /> lubiz | 0.2147 0.2518 0.4319 0.0699 1.0000 <br /> rodriguez | 0.1811 0.1309 0.3219 0.0898 0.0476 1.0000 <br /> juarez | 0.4957 0.1259 0.1727 0.0109 0.4051 0.1436 1.0000 <br /><br /><br /><br />B[7,1]<br /> c1<br />r1 0.36<br />r2 0.29<br />r3 0.32<br />r4 0.22<br />r5 0.35<br />r6 0.27<br />r7 0.34El escritorhttps://www.blogger.com/profile/10020567455325195391noreply@blogger.comtag:blogger.com,1999:blog-8391293127288856260.post-26318160652806826152016-07-27T10:05:08.030+01:002016-07-27T10:05:08.030+01:00Well, here's all the correlations, from 2015 b...Well, here's all the correlations, from 2015 back to 2012. After the standard correlations I then do all the rank correlations, in the same manner. <br />What do we learn? <br />Rank correlations don't make a lot of difference. <br />Only a few correlations between judges in 2013 and 2012 are statistically significant (the * indicates significant at 5*). Hence most of it is random!<br />There was a very cohesive set of judges in 2013<br />Eduardo Masci is a real outlier!<br /><br />Might be interesting to see if judges at least agree on the best two couples (and the worst). Have to think about that. <br /><br />Long set of results follow... Aargh, comment too long. Split into sections.<br /><br />Mike.<br /><br />2015<br />(obs=41)<br /><br /> | blanco roldan delarosa torres copes cejas galera<br />-------------+---------------------------------------------------------------<br /> blanco | 1.0000<br /> roldan | 0.2409 1.0000<br /> delarosa | 0.2580 0.2577 1.0000<br /> torres | 0.2143 0.1812 0.2900 1.0000<br /> copes | 0.0647 0.2008 0.0553 -0.1512 1.0000<br /> cejas | 0.1183 0.1373 0.3761 0.2793 0.1287 1.0000<br /> galera | 0.0813 0.3913 0.2001 0.0049 0.2984 0.0621 1.0000<br /><br /><br />B[7,1]<br /> c1<br />r1 0.28<br />r2 0.34<br />r3 0.35<br />r4 0.26<br />r5 0.23<br />r6 0.30<br />r7 0.29<br /><br /><br />2014<br />(obs=41)<br /><br /> | eduard~i JorgeF~o Javier~z juliob~a VilmaV~a claude~a OlgaBe~o<br />-------------+---------------------------------------------------------------<br />eduardomasci | 1.0000<br /> JorgeFirpo | -0.0905 1.0000<br />JavierRodr~z | -0.1226 0.1511 1.0000<br />juliobalma~a | -0.0939 0.2734 0.2583 1.0000<br /> VilmaVega | 0.0448 0.1019 0.3668 0.1990 1.0000<br />claudelina~a | -0.1795 0.2706 0.4213 0.3922 0.2200 1.0000<br /> OlgaBesio | 0.0698 0.0859 -0.0244 0.2158 0.1818 0.0386 1.0000<br /><br /><br /><br />B[7,1]<br /> c1<br />r1 0.09<br />r2 0.26<br />r3 0.29<br />r4 0.32<br />r5 0.30<br />r6 0.31<br />r7 0.22<br /><br /><br />2013<br />(obs=40)<br /><br /> | zotto martin~y ermocida schapira roldan brufman garcia<br />-------------+---------------------------------------------------------------<br /> zotto | 1.0000<br /> martinezpey | 0.3627 1.0000<br /> ermocida | 0.6356* 0.4795* 1.0000<br /> schapira | 0.2730 0.2537 0.1477 1.0000<br /> roldan | 0.3086 0.2274 0.2218 0.5632* 1.0000<br /> brufman | 0.4436 0.2098 0.1779 0.3372 0.5321* 1.0000<br /> garcia | -0.0559 0.1322 0.0787 0.0997 0.2217 0.1795 1.0000<br /><br /><br /><br /><br />B[7,1]<br /> c1<br />r1 0.42<br />r2 0.38<br />r3 0.39<br />r4 0.38<br />r5 0.44<br />r6 0.41<br />r7 0.24<br /><br /><br />2012<br />(obs=41)<br /><br /> | nieves besio balmac~a duplaa lubiz rodrig~z juarez<br />-------------+---------------------------------------------------------------<br /> nieves | 1.0000<br /> besio | 0.2979 1.0000<br /> balmaceda | 0.1973 0.2151 1.0000<br /> duplaa | 0.2959 0.1449 0.1307 1.0000<br /> lubiz | 0.2593 0.2949 0.6150* 0.1382 1.0000<br /> rodriguez | 0.1362 0.1603 0.3847 0.1180 0.1486 1.0000<br /> juarez | 0.4649* 0.1342 0.2087 0.0928 0.3948 0.1595 1.0000<br /><br /><br /><br /><br />B[7,1]<br /> c1<br />r1 0.38<br />r2 0.32<br />r3 0.39<br />r4 0.27<br />r5 0.41<br />r6 0.30<br />r7 0.35<br /><br /><br />El escritorhttps://www.blogger.com/profile/10020567455325195391noreply@blogger.comtag:blogger.com,1999:blog-8391293127288856260.post-50316732582740980532016-07-27T00:02:07.916+01:002016-07-27T00:02:07.916+01:00I've now worked out how to visualise the non-c...I've now worked out how to visualise the non-correlation between any two chosen judges in Power BI with a scatter chart. It needs some refinement, though. Calculating the Pearson correlation for the rankings turns out to be much harder in DAX than just using the scores, because you have to tell it exactly how to add up the results of RANKX, but it seems worth doing because it would be totally re-usable. Working on it!msHedgehoghttps://www.blogger.com/profile/05719152265628932122noreply@blogger.comtag:blogger.com,1999:blog-8391293127288856260.post-33417315380831410122016-07-26T19:59:33.023+01:002016-07-26T19:59:33.023+01:00Hi again,
I only posted the ordinary correlations...Hi again,<br /><br />I only posted the ordinary correlations but I did try the rank correlation too - it didn't make much difference, so didn't post.<br /><br />I'll post the results for earlier years later on (about to go and dance!). Doing it quickly, it seemed there was more agreement between judges in the earlier years.<br /><br />Mike.El escritorhttps://www.blogger.com/profile/10020567455325195391noreply@blogger.comtag:blogger.com,1999:blog-8391293127288856260.post-36372237412252823892016-07-26T14:50:27.392+01:002016-07-26T14:50:27.392+01:00@Mike - Great! Just the sort of comment I was hopi...@Mike - Great! Just the sort of comment I was hoping for. In my anxiety to get this out there, after all the faffing about, I didn't get as far as measuring the disagreement more rigorously. There's a lot to say about it, and it might be worth a part 2. <br /><br />As the judges don't seem to be awarding marks around any common average - some of their lines are just higher than others - I feel like we should measure the correlations between the rankings, rather than the scores themselves. Which did you use?msHedgehoghttps://www.blogger.com/profile/05719152265628932122noreply@blogger.comtag:blogger.com,1999:blog-8391293127288856260.post-87209657598568925852016-07-26T09:37:10.499+01:002016-07-26T09:37:10.499+01:00Hi,
The inner geek in me couldn't resist tryin...Hi,<br />The inner geek in me couldn't resist trying this out. Here is a measure of the agreement between judges. It shows the pairwise correlations between judges, over the 41 couples (2015 data):<br /><br /> | blanco roldan delarosa torres copes cejas galera<br />-------------+---------------------------------------------------------------<br /> blanco | 1.0000 <br /> roldan | 0.2409 1.0000 <br /> delarosa | 0.2580 0.2577 1.0000 <br /> torres | 0.2143 0.1812 0.2900 1.0000 <br /> copes | 0.0647 0.2008 0.0553 -0.1512 1.0000 <br /> cejas | 0.1183 0.1373 0.3761 0.2793 0.1287 1.0000 <br /> galera | 0.0813 0.3913 0.2001 0.0049 0.2984 0.0621 1.0000<br /><br />The correlations are generally low (none are statistically significant at 5%) and Copes and Torres seem to have opposing views! Copes stands out as disagreeing more with the other judges. Here are the average correlations for each judge (against the other six:<br /><br />r1 0.28<br />r2 0.34<br />r3 0.35<br />r4 0.26<br />r5 0.23<br />r6 0.30<br />r7 0.29<br /><br />Copes (row 5) has the lowest score. A couple of interesting questions: (1) is the correlation higher between judges who themselves have similar dance styles? (2) In psychology, it's known that judges become more severe over the course of a day. Does the same apply here?<br /><br />Great post!<br />MikeEl escritorhttps://www.blogger.com/profile/10020567455325195391noreply@blogger.com