Surprising economic indicators

In the article “New Ways to Read the Economy” in the Wall Street Journal  last week, journalist Cari Tuna quotes several economists who are looking at new and surprising data sources to determine levels of economic activity.  For example, Ted Egan, the chief economist in the San Francisco controller’s office, looks at Saturday passenger counts from the subway station near the Union Square shopping district to get an idea of how the local economy is doing.  And Edward Leamer, an eonomist with the University of California, looks at diesel fuel sales on California’s I-5 interstate highway (a major lumber trucking route) as an indicator of construction employment.

It's heartening to hear about people using data like this to solve problems at a time when we hear a lot about the "tsunami of data" going unharvested. (See blog entry on this).

"SuperCrunchers" referenced in Daily Beast article

An article in the Daily Beast, "How Visa Predicts Divorce" , discusses how credit card companies use data mining techniques to determine if a person is more likely to get a divorce. The author of the article mentions that this technique was discussed in the book SuperCrunchers, which I reviewed in this blog a while back.

A snippet from the article: "By scrutinizing your purchases, credit companies try to figure out if your life is about to change—so they’ll know what to sell you."

Bing and baseball

The latest post on the Microsoft Search blog describes how Bing adds value to the results of Major League Baseball related searches. Search for your favorite team and you will see the latest win/loss results, a schedule for upcoming games, and a one line summary of standings. Bing also provides real-time results for current games. Search for your favorite player and you will see a picture along with a table of their stats for the most recent game and a one line summary of their stats for the season.

In comparison, when you search your team in Google, you will see an inning scorecard of the most recent game. Also Google currently has nothing comparable to the MLB player results offered by Bing.

iPad and Gutenberg

For a thoughtful rumination on the possible effects of digital media on society (in light of the recent release of the iPad), see today's entry in the Paper Cuts blog at the New York Times.

There's also another article in the same vein from the Arizon Republic entitled "iPad, Kindle and other e-reader technology book business."

Google Public Data Visualization

In a previous post, I mentioned Google's public data feature, which allows users to explore publicly available data sets from sources such as the World Bank.

Now Google has amped up this feature with some new visualization tools. Google calls this set of tools the "Public Data Explorer," and states that now "students, journalists, and everyone else can play with the tool to create visualizations of public data, link to them, or embed them in their own web pages."

You can read the announcement from Google here.

Going to see Tufte

Edward Tufte, pioneer in the field of information visualization and author of such books as "The Visual Display of Quanititative Information," will be coming to Philly to do one of his famous one day courses. I registered as soon as I saw he was coming.

Not to be a shill for the guy, but here's the information on his upcoming courses

I'll let you know how it goes.

SuperCrunchers by Ian Ayres

There is a lot of data out there.

This fact has urged a few related notions to propagate from the realm of research into the world of business and the popular consciousness. The first such notion is that the amount of data out there is a bit frightening – there is a tremendous, staggering amount of it residing in the digital storehouses of government and industry, and it is growing bigger every minute. The second notion is that there is a desperate need to harness this data both for the betterment of society and the growth of business. The third is that this desperate need is starting to be met by emerging disciplines, tools, and methods.

I can tick off several books that reside in one or more of the above categories. “Beyond the Tsunami,” an online book from Microsoft Research mentioned in a post in this blog, is one such book. “Visualizing Data,” a title from O'Reilly Press, falls into the the third category.

But the book I want to talk about now is “SuperCrunchers,” a 2007 book by Ian Ayres, a professor at both the Yale School of Law and the Yale School of Management.

SuperCrunchers is a book about how statistical analysis of large datasets (that is, “supercrunching”) is beginning to profoundly affect our lives. The book illustrates this effect with a series of anecdotal case studies. In one example, a computer science professor started the website farecast.com after he was annoyed by learning that the people in the seat next to him on a plane had paid less for their tickets simply by waiting longer before buying them. Farecast is a site that predicts when tickets for your desired flight will be cheapest.

Ayres also describes the application of statistical methods to societal problems in the fields of medicine (evidence based medicine) and education (direct instruction). The sometimes innovative and sometimes creepy use of such methods by large corporations are detailed by Ayres as well.

The anecdotes contained in this book are compellingly written, and since the book is for a popular audience, the statistical methods described are described easily digested by the average reader.

This is an important book—-it clearly describes how the abundance of data and the statistical methods to manipulate it are changing society. This change is sometimes clearly for the better, and sometimes less so.