Data Intelligence, Cognitive Bias, and Emerging Tech  

You don’t always realize at the moment of a new technology’s introduction that it was designed to take your job.

I remember my early career working as an analyst grunt on Wall Street. To learn my job, I would compete with other analysts on a fun game called merger practice. We’d be expected to create a fully merged Profit and Loss statement for two merged companies in 15 minutes, or less.

I had to work between machines. I’d look at the Bloomberg terminal’s company data, piped real time through some sort of non-Internet based system. I’d toggle my head over to my PC, where I’d copy the numbers into Excel to show how two companies would merge together. We’d issue reports fast, faster than other analysts. That was our edge.

Then one day, it was all connected. Market data piped right into Microsoft Excel. It was a true human-centered achievement. Hooray. Time saved. Lives better. I no longer had cramps from all of that neck rotating.

In fact, soon I would not be needed. Who needs a grunt to merge companies fast when that can now be done with a button?

I was lucky. I self-selected out of Wall Street and found myself in more creative and swashbuckling environments like tech startups and design firms and then my own companies after that. I do business modeling for a living, but insist on all of the fuzzy work that needs to happen with humans before we make spreadsheets. I stayed just ahead enough of technology to keep me from being replaced by a button.

Just this week, I saw some tech that called me back to those analyst days, and made me hopeful for data analysts, everywhere.

I was judging the startups at Strata Hadoop World in NYC, the data conference.

At Strata, everyone is an amateur behavioral economist. Humans have bias. Humans aren’t rational. Humans also don’t scale as fast as machines.  In the tutorials and main stage shows, everyone is trying to learn how to use the tools of data science to design human fallibility out of the system.

Startups are focused on delivering automation, to scale faster. Reproducibility was a  big theme, because of all of the pesky human error.

For example, Compellon, the startup who came in second place, seeks to overcome what it calls the trial and error approach to data analytics. The company’s analysis engine aims at eliminating the traditional analytics method of testing data against multiple statistical models. The promise from Compellon: “Not a statistician? Not a problem. Compellon 20|20 was built to deliver answers for business professionals with or without data science skills.”

There’s an equal and opposite trend happening at Strata Hadoop and the edges of technology culture. Tim O’Reilly, the CEO of O’Reilly which runs that Strata conference series, has called innovators to augment humans. “Don’t Replace People. Augment Them. If we let machines put us out of work, it will be because of a failure of imagination and the will to make a better future.”

At the Startup Showcase, a company called Virtual Cove is augmenting financial traders. Virtual Cove is creating mixed reality (augmented reality, virtual reality) to let you see thousands of data points at one time. “In contrast, Virtual Cove’s approach lets your mind absorb combinations simultaneously.  This way, the lock simply opens, insight intuitively revealing itself.”

Once you put on the headset, you can see the entire Nasdaq stock market and 100 of the top companies, each with hundreds of data points. You can get a sense of the visualization in this 2-D image below – but the real benefit is being able to walk around and quickly take in all of that information.

The founder, Bob Levy, points out that that humans have skills that are hard to reproduce in machines. The role of the trader has been decimated by the rise in algorithmic trading. Code and software and data have replaced humans, because they move faster. But the ability to quickly take in sensory data in 3-D form and process all of that information visually could be our saving grace.

Where else could this go? Could we move beyond augmenting our analytical brains and tap into our human desire to feel, to express, to play?

Having had the pleasure of playing with Tilt Brush on the HTC Vive, I could envision a future virtual trading floor of traders hurling iridescent company stock sculptures at each other to make a trade.

I could see a future annual shareholders meeting, with community activists and eco-activists parading their data sculptures of impact in front of the company’s financial performance review.

What do you see as the future of data intelligence? Do you see machines gunning for your job? Are you at risk of being automated, or will you benefit from tech and data-driven augmentation? How are you playing with new technology make a better future?