Saturday, January 11, 2014

New ways of representing data

The New York Times Bits blog this week has an article titled "A Makeover for Maps" that talks about cutting edge visualizations of data.  The article cites some interesting new depictions of data--such as the one created by Eric Rodendeck of  Stamen Design that creates  "...a representation of how photos spread on Facebook that looks like ice crystals forming on a car window."

The article goes on to talk about the idea of creating visualizations that can be used by different audiences -- like a dynamic chart of Nasdaq financial data that can be read one way by a pension fun manager looking for trends and another way by an SEC investigator looking into an unusual spike in trading during a given day.

Check out the full article with multiple visualization examples at

Monday, October 14, 2013

Looking at the problems on the site

You may have heard the news about people in the U.S. experiencing problems trying to sign up for healthcare on the new site.  This site is perhaps the most visible manifestation of the Affordable Care Act, aka "Obamacare."  On, users can search the "Health Insurance Marketplace" for coverage.  However, the site has been plagued by reports of slowness and poor user experience. 

Several online articles attempt to surmise what is going wrong with the site from a technical perspective.  Regardless of your personal views on the Affordable Care Act, I think these are relevant for us to read because they provide us a chance to avoid making the same (apparent) technical mistakes in our projects.  I'd like to highlight some general points that I have read so far:

  • Load test both the interface _and_ the back end functionality. 
  • User experience (UX) should not be an afterthought
  • Poor validation logic (eg requiring numbers as part of username/password but not informing user of this beforehand or in error messages) leads to extremely poor UX

For more on the technical issues with, see the following articles:

And here is a contrarian opinion, stating that the architecture of is sound, but that the project was rushed:

Friday, September 27, 2013

Google updates its search algorithm

Google has updated its search algorithm for the first time since 2010.  According to this post in the Bits Blog on the New York Times site, this is the most extensive overhaul of the algorithm since 2000.

Although Google is not revealing specific details of the new algorithm, code named "Hummingbird," the main thrust of the changes are geared toward understanding longer and more complex queries.  To that end, Google is employing its Knowledge graph, which is a map of semantic relationships.

One of the drivers for understanding longer queries is that more people are speaking queries into their phones using natural language, which is by its nature more complex than a simple keyword query typed into a search box.

For more information, see:

Forbes article:

Search Engine Land FAQ on Hummingbird:

Google "15th anniversary" blog post (includes a great "Google Search Timeline" graphic):

Google post on the Knowledge graph (2012):

Saturday, May 11, 2013

YouTube trends map

YouTube Trends map

YouTube has a new feature called "Trends Map" that allows you to see what people in different parts of the country are watching.

Here is the YouTube trendsmap site:

And here is an article describing the new feature:

Monday, April 8, 2013

"Predatory journals" in science

Interesting NYTimes article today about a disturbing phenomenon in academic research:  "predatory" journals and conferences that charge (unadvertised) fees for publication.

Tuesday, February 5, 2013

Friday, November 16, 2012

The Role of Data Analysis in "Get Out The Vote" Operations

In this article in The Atlantic, Ben Jacobs describes the failures of the Romney campaign's "ORCA" software platform.  ORCA was supposed to help the campaign identify which likely Romney voters had already cast their ballots.   Thus, they could focus get-out-the-vote operation resources on voters who hadn't gone to the polls yet. 
It turns out that the Obama campaign had tried something similar in 2008, and that system had also crashed.   The Obama campaign learned its lesson from this experience. They came to the conclusion that such an election day system was not feasible.   For 2012 they went with a system called "Narwhal" that targeted likely voters before election day. 

So in a sense the Romeny campaign was on cycle behind from a technical perspective -- they learned a lesson that the Obama campaign had learned 4 years earlier.