Workforce Planning for Data, Technology and Analytics Teams
September 20th, 2022
This November we celebrate 18 years of helping businesses and individuals within the data, technology and analytics space grow their teams or career. Over that period so many changes have taken place within this fast-moving sector, both within the sector and the working environment. The timeline below shows how far the sector has grown.
Doug Laney, Gartner coins the 3V’s of Big Data (Volume, Variety and Velocity) to help define the properties of big data.
Google announces its plans to float on the stock exchange.
Facebook is launched with one million users by the end of its first year.
Web 2.0 increases data volumes. The birth of HTML style websites with SQL back-end databases means more user generated content adding to the increasing mountain of data in this space. Apache Hadoop is created by Doug Cutting and Mike Cafarella.
Amazon Web Services is created, offering web-based computing infrastructure services. 18 years on it now dominates the cloud services industry with roughly one third of the global market share.
Google Chrome is released with support for 43 languages. 3G is launched for faster data flow. GPS goes mainstream and now getting lost is no longer an option. Apple App store is launched making it easy for mobile app developers to get their software into the hands of the users.
McKinsey reports that there will be a shortage of 140k-190k of Data Scientists by 2018 in the US and 1.5 million Data Managers.
Google Brain releases the findings from the so called Cat Experiment where a neural net spanning 1000 computers was shown unlabelled images from YouTube. The training software was set to run after being shown the images and one neuron in particular was found to react strongly to images of cats. The Cat Experiment was around 70% more successful than its forerunners in processing unlabelled images. Gartner predicts Big Data demand will reach 4.4 million jobs globally by 2015 but only one third will be filled.
DevOps becomes mainstream, based on the idea that the development and operational teams work in a more cohesive, transparent end-to-end way, with the shared goal of successfully delivering completed products or projects. Cultural developments are needed for this role to work successfully. Backend as a Service (BaaS) increases in popularity.
Data Visualisation becomes more popular as a way to translate data into usable insights. Predictive analytics is in its infancy and Amazon joins forces with Microsoft in offering a platform to use Machine Learning in teaching AI entities. The new platform, called Gluon, grants access to AI developers of all skill levels. The open source Gluon platform was described as making it easier for AI developers to design and develop neural networks and was set up on Amazon Web Services.
Covid-19 hits and many companies moved quickly to enable remote customer interaction and service, while data emerged into an even greater light of importance, visibility, necessity and urgency. Data from disparate places were being pulled together by Governments, Universities, Healthcare providers and industry to understand the virus better. Home working becomes the norm for many of the population.