The 2006 World Cup Final in Berlin. (Photo city vhs wm/Fickr)

View full size photo »

Since Friday's World Cup draw, you've read and heard a lot of opinions about which groups are strongest, weakest, top-heavy or E.A.S.Y.  It is by intention that you have heard little of those opinions here (though I was not above doing that at other places, like here and here).

To be honest, I believe good analysis is done in spans, not moments, and I did not feel comfortable posting my knee-jerk reactions.  In the great comments threads (here, here) we had in the wake of the draw?  No problem.  That's a different context.  For a post, however, I wanted to be deliberative.

At least for this topic, I wanted to be deliberative.

This weekend, I spent a lot of time talking with other analysts (read: my family - it is the holidays), reporters (bill collectors), fans (the counter help at Lefty's, the pizza place down the street), as well as contemplating any holes in my own analysis (otherwise known as "talking to a 22 oz. of Sapporo").  Now, I'm ready (if a bit dehydrated).

Over the next four days, I will be looking at each of the groups, talking about each nation's strengths and weaknesses, using that to inform where I feel each quartet falls on the spectrum between "Group of Death" and "Tribe of Life."  Later today, I'll roll out my views on the Tribe of Life - the draw's easiest group.

But before I start that series, I wanted to share a small part of my research, comparing of a number of "quantitative" views of team strength.

I say "quantitative" because one of the four systems I use is a poll: the aforementioned Set Piece Analysts poll.  The other three measures are the December FIFA World Ranking, the latest Elo Rating, and the new Soccer Power Index at Soccernet (developed by Nate Silver of PECOTA, Baseball Prospectus and FiveThirtyEight.com fame).

While it would be helpful if I took some time to explain how each of these ranking systems work, that will need to wait for another time (soon, though).  What's important for today's analysis is the differences in output.  Limiting ourselves to qualified teams:

• Elo's rating system has a highest rating of 2081 (Brazil) and a low of 1508 (South Africa).
• FIFA's ratings go from Spain's 1622 to South Africa's 377, a much broader range.
• The Soccer Power Index yields a high of 88.6 for Brazil to North Korea's low of 56.9, a much more narrow numerical range.
• Set Piece Analyst's highest point total is 286 (Brazil) but goes all the way down to 9 (New Zealand).

First, a reminder about the group composition, which will be referenced throughout this piece:

With that in mind, we can start asking some questions. First, according to each measure, which group is strongest?

I took the ratings for each team, by each measure, and figured out the average (mean):

Amongst the quantitative systems, there is agreement.  Group H is the toughest group, a conclusion that runs counter to almost everybody's weekend analysis.  Though we've heard talk of Groups G and D and even a little about E, few are thinking Group H is the toughest group despite the average rating of Spain, Switzerland, Chile, and Honduras being the highest (according to our three rating systems).

What are the systems missing that the analysts aren't?

It isn't so much what the systems are missing; rather, it is what they are dwelling upon.  Consider the FIFA rating, where Spain and Brazil are each over 300 points ahead of the third-place Netherlands.  When you use a statistical mean to look at a group, an outlier can end up skewing the measurement.  Spain and Brazil are huge outliers by the pure FIFA rating, and that's part of the reason groups H and G are one-two per FIFA (but no other system).

Similar effects can be seen within Elo.

The only group that is in the top three of all four measurements:  Group E, consisting of the Netherlands, Cameroon, Denmark, and Japan.  Group E finishes second with three of our systems.

On the other end of the scale, Group F fares the worst, and it's not very close.  By all three analytics, they are markedly behind seventh place.  Again the Set Piece Analysts see a different world, with Group A being their weakest quartet.

To make the numbers a little easier to look at, I reproduce Table A (below), but instead of producing the ratings, I give you the ranking.  This will allow you to look at a group (say, H), their ranking (say, 18.5, by FIFA) and correlate that with a team (which would be Switzerland, per FIFA).

Or, more directly, you can say "the average team in that group is about 18th in the world?

"Seems like a tough group, for a 32-team tournament."

However, the table is not exactly the same. Notice that the order changes when you look at the teams by rank instead of rate.

If we're still looking within the individual systems, why does the order change when we switch from rating to ranking?

Consider our Spain example. Spain is over 300 points ahead of the Netherlands in the FIFA world ranking, about the same point distance between the third-place Dutch and fifteenth place Mexico. When you go by ranking, the distance gets narrowed to  a flat "two."  Two places from Holland in the FIFA ranking: Portugal.

Again, we see Group E serve as the consensus, tough group, though H is still favored by two measurements.  Group F still looks easy.

As we move through the upcoming week, with my group analysis being posted, I will be referencing these measures.  When I talk about Group E (and that will obviously be later in the week), I will reference their Elo, FIFA, SPI, and SPA "ranks" to give everybody an idea of what The World Beyond Richard (as I like to call it) thinks of the groups.

There are two more tables I want to share with you, but I will warn you:  a lot of you will want to tune out at this point.  I'm going to bring in a simple way of measuring variance (under the guise of assessing group depth), and while it's neither complicated nor something that's beyond the people who read this blog, it is a level of analysis that turns a lot of people off.

Consider that your disclaimer.  

Now, let's look at each of the groups in terms of the standard deviation.  I present Table C unordered, by rating (not ranking), and I'll go into the implications of using standard deviation, below the table:

As with the ratings, the standard deviations of the groups' ratings are meaningless when comparing across systems, but when you look at the systems in isolation, you get a good idea of the relative balance of the groups.  Looking at Elo, Group B is by far the most balanced, with Group G being the most unbalanced.

How does that work? A standard deviation is a quick measurement of how much the groups' values (data points, observations) differ from the group's average.  If you have a group with an average rating of  50, the ratings of the four members could be 1, 2, 98 and 99, or they could be 48, 49, 51, and 52.  Those are two entirely different groups, but their average (mean, or also median, in this case) is the same: 50.  

The standard deviations of those two groups would be wildly different.  The first group would have a high value, while the second's would be low.

Standard deviation is telling us that, per Elo, Group B is very balanced.  Of course, FIFA sees Group C as the most balanced.  Just because we're getting fancier with our measurements doesn't mean we're going to reach an accord.  In fact, SPI sees a different group as most balanced:  Group D.

While standard deviation gives us a clue about balance, it is not a good measurement when you're trying to determine more than mere parity, and I want to use it to give us some idea about group strength.  To do that, I need to reference the group's average.  Standard deviation is a measurement relative to that average, but it does not care whether that average is a high or low value.  By coming up with a measure that takes the average into account, we can get a very idea of quality and depth.

In our final table, I take the averages from Table A and subtract the deviation.  This usage is anything but scientific, but if gives us a loose idea of a statistical floor - a minimal quality of the group.

(For those statistically inclined:  a.) I apologize, b.) using the rigor of z=-2 as the floor is not the purpose, as the concept of a Group of Death is inconsistent with considering such a large conceptual range.)

Here are those results:

Since we're relying on variance, this measurement will not work with rankings. We have to look at the ratings.

Again, Group F looks like the easiest quartet, and even when we switch to this new measurement - designed to tease out some of the outlying ratings - Group E looks like the deepest, by consensus.

Of course, the surprise by this measure is the FIFA columns.  Group C, when you use this measure of quality and depth, comes out on top?  I would caution you to think both about the quality of teams like Slovenia and Algeria as well as the limits of this method and the quality of FIFA's ranking.