I've had a non-technical topic on my mind for some time. Well, a non-database topic, anyway. As one who enjoys digital photography, I use my computer to prepare images for publishing. I mainly use Adobe Lightroom, Photoshop Elements and a few add-in programs. I find working with my images fun and surprisingly cathartic. 

Those who work closely with me understand that "Big Data" is not my favorite phrase. The hype is certainly big, even if the specifics are not. Semantics aside, there's a lot to love in the world of data right now. The way that we THINK about using data is definitely changing, or at least being refined. We have to prioritize what data we're going to analyze. (Hint: start with what's available...) HOW we analyze and consume data is really driving the true change.

I earned my very first paycheck from my first real job at McDonald's. $3.03, net of taxes. I learned a lot in those nearly two years of working side by side with Ronald McDonald and the Hamburglar. I recall a certain amount of curiosity at how much measuring was taking place. I'm not referring to the amount of lettuce on a McDLT or the number of ice cubes in a large Sprite. During some shifts, it seemed that all some managers did was fill out reports -- product usage, product waste, drive-thru times, etc. I did not envy whoever got the mundane task of entering those handwritten reports.  

Columnar databases have been kicking around the DB world for well over a decade, and the topic has gained momentum in more recent years. While not suitable for OLTP systems, they do offer some exciting performance opportunities in data mining scenarios when data updates can be managed. For an overview of columnar databases and the expected advantages, I like the following brief article from Ralph Kimball:

A few weeks ago I attended the Analytics 2012 conference sponsored by SAS. I enjoyed much of the content and want to highlight just a couple of sessions.

I recently returned from an overseas trip and passed some of the flight time by reading additional documentation of the software packages, JMP. As a bit of background, to date I have focused my use of JMP to the  "Analyze" menu — standard x by y distributions, regression, partitioning, etc. For my graphing needs, I currently prefer using Tableau to JMP's graph builder. As for the "DOE" menu? I had never used it. With the dedicated flight time, I dug into the 350+ PDF "Design of Experiments Guide."  I worked through a few of the samples, step-by-step.

Big Data and its co-conspirator —  the data scientist — are grabbing a lot of headlines these days. To be sure, the piles of data available for analysis and true mining for value are enormous. Petabytes, exabytes, whatever. Search for articles on the subject, and you'll find mentions of Twitter, Facebook, Yahoo, LinkedIn, and the other usual suspects. I had a conversation the other day discussing the amount of data in daily extracts from DNS servers. Lots of fascinating activities. 

The economic principle of scarcity explains the dilemma of matching scarce resources against unlimited wants. Customers, clients, projects managers, etc., typically provide the unlimited wants. If I'm honest about it, I'm just as guilty about MY unlimited wants.: tuning another report, scoping a a new feature, learning a new technology, working to solve another problem, all seem to compete for time.