Too much information

In many realms, a lack of data can be a critical blocker which hinders progress. A field where copious amounts of data is typically advantageous is machine learning, more data can generally improve results. For us though, an abundance of data can mean something very different and what’s more there is no shortage of data out there. Never before has data and information been more readily available.

The explosion of data and information availability is largely due, in part, to the internet. The availability of internet-based platforms and services like Amazon Kindle, Audible, Coursera and YouTube have meant that information and knowledge have been democratized like never before. As I’ve said to quite a few people, YouTube and reading books has probably taught me more about engineering than university has. Being able to consume information in this way is just easier and, in many ways, more engaging.

With that being said, there is still a fundamental problem with the information availability explosion and that is the finite time available to consume and process it. The first time I really considered this problem was a few years ago when I was walking around a bookshop in Belfast looking for a new book to read. I don’t know where the thought came from but it just popped into my head, “There are far too many books to read in this bookshop for a lifetime”. It dawned on me that even if I dedicated my entire life to reading (which isn’t very realistic), I still wouldn’t have enough time to read every book in that shop. What’s more, hundreds of new books are being published every month so this is definitely not a stationary problem, we are quite literally drowning in the information that we produce.

Armed with this insight, I realised that accessing knowledge and information was only half the battle, knowing what information to attend to and when was the real challenge I faced. I’m still trying to figure out a solution to this problem but as I’ve said before it’s clearly a dynamic one. As our goals and interests change, our ways of identifying and consuming information must also change. As I think about it more and more, I’m coming to the conclusion that the solution is part skill/process but also part instinctual. My belief is that it may be like the ever-changing financial markets, in reality anyone can open a brokerage account and buy shares on the exchanges, but the real skill is knowing what shares to buy and when to be able to generate those significant returns on investment.