Simply put, data-driven businesses analyse and use their internal and external data to influence strategic decision making.

“Every day a whopping 2.5 quintillion bytes of data are created”


But what are businesses moving to becoming data driven? According to an IBM study, 90% of the world’s data has been created within the past 2 years. Every day a whopping 2.5 quintillion bytes of data are created.

Just to give you an example (and some insight into my life too), my normal morning routine, which is probably not so different from most, generates several sets of data: checking Facebook and Twitter when I wake up – which will both capture and record this as an activity; putting the toaster on – which my electricity company will make a record of; using an app to pay for the bus – which generates data that is captured on my phone and within the bus company’s systems; checking online banking – again captured on my phone and my bank will also record exactly what I did and when and then, of course, getting my coffee – which again my bank will know of because I’m paying by card and my loyalty app will make an entry of this too.

… And this is just before I get into work, all of these data points are captured and stored with the potential to be analysed for insights and trends.

“Forward-thinking companies will be able to use their data repositories to understand and target their customers better”


With figures like quintillion bandied around, the term “Big Data” has emerged to define the huge data stores that companies are now collecting and storing, often with rows of data going into the billions. These new “Big Data” sets need specialised and distributed technology to be able to store, cleanse, access and analyse this data.

So what does this mean for businesses?

There is a wealth of data being generated, collected and stored, but without the means to be able to recall and analyse this data, it has no use to businesses. Businesses that don’t start to utilise this data will be left behind by their competitors. However, ‘utilising data’ needs to be strategic and data-sets are not generated and captured in a nifty dashboard format, hence businesses need to ensure that they have the data infrastructure and technical staff in place so that the value that lies within all this data can be realised and utilised strategically.

“By being able to hyper-personalise services and goods, companies will remain relevant and current”


This is not news, businesses are aware and onto it. Big data adoption across FTSE 250 firms reached 53% in 2017, up from 17% in 2015, with telecom and financial services leading early adopters.

How can companies use this to benefit their business?

Forward-thinking companies will be able to use their data repositories to understand and target their customers better. It’s all about making sure what you offer is and remains relevant and what your customers choose and want. It’s about relevancy. In an age where customers have more information about prices and services at their fingertips, where clients can make educated decisions based on their own research online (think reviews etc.), the ability to better target your customers will lead to increased market share. As you will be able to truly be present with your offering when they are most likely to care.

“Internal business data generated can help to understand more about internal business operations”


By being able to hyper-personalise services and goods, companies will remain relevant and current in an age where customers are bombarded by constant offers, emails and promotions. Only the businesses that are targeting customers accurately will succeed.

On top of this, if companies have enough data and have cleansed and prepared it, they can build algorithms and train them with this data to start predicting customer demand and retention or predicting when machines may break down or identifying correlations between data sets.

Beyond the above, internal business data generated can help to understand more about internal business operations. Analyzing them and implementing change based on information can improve efficiency and reduce costs.

Why are businesses now starting to look at big data solutions?

Previously businesses needed physical data storage in the form of servers or space in data centres. This had associated costs and potentially staffing costs to administer these systems.

Also, historically there’s been a shortage of analytical and technical staff to build cloud-based systems and to analyse the data they contain.

“Experienced data scientists and engineers command salaries upwards of £60,000 with just 2-3 years’ experience”


Fast forward to now, the price of cloud computing has reduced significantly meaning companies can now keep their data in the cloud rather than in physical data centres. This allows scalable ‘pay as you use solutions’ and they can also store much more data than they could previously.

And skills have evolved from those developments, there are now more people out there that are able to analyse this data to provide the insights and trends needed to improve their businesses.

So what new jobs are being created as a result of all this data?

Most businesses already have the surface-level skills to analyse the data available to them. For example, businesses can measure websites using Google Analytics and Facebook visits with Facebook Analytics. They can record and recall internal data on MS Excel spreadsheets or Google Sheets. Businesses record their customer data on CRM platforms such as MS Dynamics and Salesforce. They can build basic charts and graphs using MS Excel or MS PowerPoint.

The kinds of jobs that are being created, allowing businesses to become data-driven include:

  • Data Engineers – the people that build your databases, data warehouses and data lakes.
  • Insight Analysts – the people that will statistically analyse your data to discover trends and correlations.
  • Data Visualisation Experts – the storytellers that will paint a picture with your data to show what is happening.
  • Data Scientists – the all-rounders, a better statistician than any programmer and a better programmer than any statistician. They devise and run experiments to solve business challenges.
  • Data Wranglers – clean, mine and organise data so it can be analysed.
  • Machine Learning Engineers – programmers that design and build algorithms.
  • Deep Learning Researchers – those looking into the cutting edge of what is possible with algorithms, neural networks and beyond.

However, as businesses start to understand the value of data, such skills and talent are in high demand – demand above the supply.

At present there is a massive shortage of experienced candidates with the commercial exposure needed to work on data science and analytics projects and to build out data infrastructure.

This shortage is driving up salaries across the South West in this space with experienced Data Scientists and Engineers commanding salaries of upwards of £60,000 with just 2-3 years’ experience.

So how can we get ready for this future, what skills and academic subject knowledge are needed for these new jobs?

Academic subjects that are useful for careers in data and analytics are STEM (science, technology, engineering and maths) subjects, however, these are not the only routes into data and analytics jobs.

“There is an abundance of online courses that are recognised by employers”


Marketing, journalism, media, communications, graphic design, business, geography and linguistics all have relevant transferable skills that are becoming useful in this new data age.

From customer behavioural analytics and data visualisation to GIS analysis and speech recognition – there are a whole host of use cases for this technology which benefits from these skill sets and knowledge.

What are the technical skills required for Data & Analytics jobs? In short:

  • Data manipulation tools – SQL, VBA
  • Statistical programming languages and tools – R, Python, SPSS, SAS
  • Data visualisation tools – Tableau, ClickView, Microstrategy, Power BI
  • For Machine Learning / AI – programming languages –  Python, Java, R, C++

How can candidates gain the skills needed for these new jobs?

A-Levels, vocational and college qualifications, university degrees – formal education in STEM subjects or transferable subjects such as media, marketing, design, geography are a good starting point.

Beyond and in addition to that, there is an abundance of online courses that are recognised by employers such as Coursera, Udacity, EDX, IBM, Dataquest and KDNuggets.

Online resources such as Kaggle can support beginners in a whole range of things from courses to mentors to practice data analysis exercises.

Also work placements, internships and research collaborations can really help students understand what it is truly like to work in this field and can provide them with the invaluable commercial experience needed for these jobs.

What do you need to offer analysts to retain them?

Investment in technology and tools. Candidates in this space want to work using modern technology that will benefit their career long term, investing in modern data analysis and visualisation technology will get these candidates excited.

Training budgets. This is a constantly evolving space, new technologies and tools come out all of the time. To ensure you can attract and retain the best candidates who want to continuously learn, providing training budgets will go a long way.