Tuesday, August 2, 2011

The Heart of Haiku (Kindle Single)The Heart of Haiku by Jane Hirshfield

My rating: 5 of 5 stars

I really didn't know what to expect, and was surprised how much I liked this little book. It paints a real picture of the time and the role of one pretty unusual man, Bashō, in evolving this form of poetry.

The book has a way of connecting the past to the present. Here the author is talking about "renga" writing in 17th century Japan:

"As with Dungeons and Dragons a few years ago, or Worlds of War and Second Life today, linked verse brought its practitioners into an interactive community that was continually and rapidly evolving. Hovering somewhere between art-form and competition, renga writing provided both a party and a playing field in which intelligence, knowledge, and ingenuity might be put to the test. Add to this mix some of street rap’s boundary-pushing language, and, finally, the video images of You-Tube. Now imagine the possibility that a “high art” form of very brief films might emerge from You-Tube, primarily out of one extraordinarily talented young film-maker’s creations and influence. In the realm of 17th-century Japanese haiku, that person was Basho."

There are some very interesting renditions of concepts hard to capture in the English language, such as "sabi". And the poetry itself, then, takes on a whole different feeling. Some of it is quite funny:

a hangover?

who cares,

while there are blossoms

Saturday, February 19, 2011

The Quick & Dirty Index (QDI) of NYT Difficulty- A Work in Progress

Computing the Quick & Dirty Index (QDI)

Here’s how to get a semi objective estimate of the difficulty of the New York Times Crossword Puzzle based on the first 50 on line solvers (So, this can be done within 10- 90 minutes of the puzzle posting):

On the Applet, Select: “First” (Not Fastest)

At the 5 min mark, note Number of People who have completed the puzzle= N1

At the 10 min mark, note Number of People who have completed the puzzle= N2

At the 15 min mark, note Number of People who have completed the puzzle= N3

Note the Time it takes 50 people to complete the puzzle: T

QDI= T/ N1+ N2+N3

Example: A Tuesday Puzzle

At 5 min: 12 people have done it- N1= 12

At 10 min: 28 people have done it- N2=28*

At 15 min: 40 people have done it- N3=40*

It took 22 minutes for 50 people to complete the puzzle, so T= 22

QDI: 22/12+28+40 = 22/80

QDI= 0.275

*NB: N2 and N3 are the actual numbers of completers as you read them from the list at each point, NOT the incremental number since the last reading!

Predicting Difficulty with the Quick & Dirty Index (QDI)

E= Easy EM= Easy-Medium M=Medium MC= Medium- Challenging C= Challenging

Monday** E: <0.15 EM:0.15-0.2 M:0.2-0.3 MC: 0.3-0.4 C: >0.4

Tuesday: E: <0.10 EM:0.1-0.14 M:0.15-0.2 MC: 0.2-0.3 C: >0.3

Wednesday E: <0.20 EM:0.2-0.25 M:0.25-0.3 MC: 0.3-0.39 C: >0.4

Thursday E: <0.40 EM:0.4-0.6 M:0.6-0.9 MC: 0.9-1.4 C: >1.4

Friday E: <1.0 EM:1.0-2.5 M:2.5-3.5 MC: 3.5-7.0 C: >7.0

Saturday E: <4.0 EM:4.0-5.0 M:5.0 -7.0 MC: 7.0-11.0 C: >11.0

** Since Monday posts at 6pm on Sundays, the scores are not comparable to the rest of the week (it looks like a Wednesday). Fewer people get on line to solve it at dinnertime (or earlier) relative to later in the evening. Also, in truth, the best index for Monday is how many people finish in the first 5 minutes. A scale for this is in progress...

Validating the Quick & Dirty Index (QDI)

QDI is being validating with the information posted by sanfranman based on computing the median time for the entire population of solvers and the 100 fastest solvers on any given day. Agreement rate is 80-90%, depending on how you squint at it.

Big CAVEAT re the Quick & Dirty Index (QDI)

If something major is going on in the world during that first 15 min-1 hour after posting, then the QDI is not valid (Earthquake, World Series, Oscars, etc.)

Similarly, the time of day that the puzzle posts is relevant. Since Mondays post at 6pm Eastern on a Sunday evening, fewer people solve during the first few minutes than on Tuesday at 10pm. So, comparisons to assess difficulty are run within a given day of the week, not between days.

Why have a Quick & Dirty Index (QDI)?

My original trigger was a comment someone made to sanfranman on Rex’s blog, saying that his data are not valid because anyone can cheat and do the puzzle on line after having solved it off line, artificially skewing the data. I responded by saying that while this is true (and some numbers that get posted seem unbelievable) the data appear generally consistent with subjective experience, day of the week, and Rex's ratings, suggesting that the percentage of people who cheat is minimal.

I also said that a good way to gauge difficulty while minimizing the impact of cheating is to look very early on after the puzzle has posted. If someone had completed the puzzle at 2.5 min. after it posted the odds of that person having cheated are essentially non-existent- That time would be higher on Saturdays, but again, if someone posted at 8 min. it would be unlikely that they had cheated.

That led me to my efforts to monitor early postings. While looking at the number of completers at 5 minutes after 10 pm is informative on a Tuesday, it is useless on a Saturday, since hardly anyone ever finishes at 5 minutes. Hence the sampling at 5,10,15 minutes, and then having a constant index of how long it takes the first 50 to complete. The latter number is quite informative but it misses some of the dynamics of whether a lot of that got accomplished rapidly or slowly. The QDI is a poor man’s way of getting a sense of rate.

Mostly, it's a geeky way to keep amused until real data are in :)