Computing the Quick & Dirty Index (QDI)
Here’s how to get a semi objective estimate of the difficulty of the New York Times Crossword Puzzle based on the first 50 on line solvers (So, this can be done within 10- 90 minutes of the puzzle posting):
On the Applet, Select: “First” (Not Fastest)
At the 5 min mark, note Number of People who have completed the puzzle= N1
At the 10 min mark, note Number of People who have completed the puzzle= N2
At the 15 min mark, note Number of People who have completed the puzzle= N3
Note the Time it takes 50 people to complete the puzzle: T
QDI= T/ N1+ N2+N3
Example: A Tuesday Puzzle
At 5 min: 12 people have done it- N1= 12
At 10 min: 28 people have done it- N2=28*
At 15 min: 40 people have done it- N3=40*
It took 22 minutes for 50 people to complete the puzzle, so T= 22
QDI: 22/12+28+40 = 22/80
QDI= 0.275
*NB: N2 and N3 are the actual numbers of completers as you read them from the list at each point, NOT the incremental number since the last reading!
Predicting Difficulty with the Quick & Dirty Index (QDI)
E= Easy EM= Easy-Medium M=Medium MC= Medium- Challenging C= Challenging
Monday** E: <0.15 EM:0.15-0.2 M:0.2-0.3 MC: 0.3-0.4 C: >0.4
Tuesday: E: <0.10 EM:0.1-0.14 M:0.15-0.2 MC: 0.2-0.3 C: >0.3
Wednesday E: <0.20 EM:0.2-0.25 M:0.25-0.3 MC: 0.3-0.39 C: >0.4
Thursday E: <0.40 EM:0.4-0.6 M:0.6-0.9 MC: 0.9-1.4 C: >1.4
Friday E: <1.0 EM:1.0-2.5 M:2.5-3.5 MC: 3.5-7.0 C: >7.0
Saturday E: <4.0 EM:4.0-5.0 M:5.0 -7.0 MC: 7.0-11.0 C: >11.0
** Since Monday posts at 6pm on Sundays, the scores are not comparable to the rest of the week (it looks like a Wednesday). Fewer people get on line to solve it at dinnertime (or earlier) relative to later in the evening. Also, in truth, the best index for Monday is how many people finish in the first 5 minutes. A scale for this is in progress...
Validating the Quick & Dirty Index (QDI)
QDI is being validating with the information posted by sanfranman based on computing the median time for the entire population of solvers and the 100 fastest solvers on any given day. Agreement rate is 80-90%, depending on how you squint at it.
Big CAVEAT re the Quick & Dirty Index (QDI)
If something major is going on in the world during that first 15 min-1 hour after posting, then the QDI is not valid (Earthquake, World Series, Oscars, etc.)
Similarly, the time of day that the puzzle posts is relevant. Since Mondays post at 6pm Eastern on a Sunday evening, fewer people solve during the first few minutes than on Tuesday at 10pm. So, comparisons to assess difficulty are run within a given day of the week, not between days.
Why have a Quick & Dirty Index (QDI)?
My original trigger was a comment someone made to sanfranman on Rex’s blog, saying that his data are not valid because anyone can cheat and do the puzzle on line after having solved it off line, artificially skewing the data. I responded by saying that while this is true (and some numbers that get posted seem unbelievable) the data appear generally consistent with subjective experience, day of the week, and Rex's ratings, suggesting that the percentage of people who cheat is minimal.
I also said that a good way to gauge difficulty while minimizing the impact of cheating is to look very early on after the puzzle has posted. If someone had completed the puzzle at 2.5 min. after it posted the odds of that person having cheated are essentially non-existent- That time would be higher on Saturdays, but again, if someone posted at 8 min. it would be unlikely that they had cheated.
That led me to my efforts to monitor early postings. While looking at the number of completers at 5 minutes after 10 pm is informative on a Tuesday, it is useless on a Saturday, since hardly anyone ever finishes at 5 minutes. Hence the sampling at 5,10,15 minutes, and then having a constant index of how long it takes the first 50 to complete. The latter number is quite informative but it misses some of the dynamics of whether a lot of that got accomplished rapidly or slowly. The QDI is a poor man’s way of getting a sense of rate.
Mostly, it's a geeky way to keep amused until real data are in :)
