A very interesting article recently caught my attention. The article, called “You are not Google” explains how often companies get carried away with the newest, latest and shiniest technologies, without first thinking whether they are actually appropriate for their case.
Some examples:
- Companies using NoSQL solutions, when a simple relational database would have been appropriate.
- Looking into using Hadoop, when the scale is such that Hadoop is completely unnecessary.
- Using new databases, that have been optimised for something else. E.g. Cassandra prioritises write availability over consistency. Is this what you are really after?
- Kafka was designed to handle huge loads (around 1 trillion messages per day). Does your business really face these loads?
Besides those issues in data engineering, we have also seen similar misunderstandings arising in data science and blockchain. A common misunderstanding is companies thinking that they can’t do machine learning unless they have huge volumes of data. While the tech giants are using technologies like deep learning with huge volumes of data, people have been doing statistics with as few as 10 data points in some cases. The amount of data you need is problem specific. If you want to build a predictive model, sometimes a few 1000s of data points can be enough. If you want to run some sort of statistical analysis, you might be able to do it with far less data.
Also, blockchain is another technology that is widely misunderstood. While there are many cases for it (E.g. in supply chains, or token economies), I’ve seen a large number of businesses where they might have as well used a traditional database to solve the same problem.
Technology can sometimes feel like a race, and it is easy to get carried away with every new technology. The situation often gets worse because of the jargon and buzzword abuse by solution providers. In the end of the day, there is an easy way to figure out whether it is a good idea to adopt a new technology. Simply use the questionnaire created by the Tesseract Academy, that helps you assess whether the adoption of a new technology is worth it or not. Don’t hesitate to get in touch if you have any questions or comments.