[IMAGE: GETTY IMAGES]
Big data is a big industry, and you don’t simply fall into a Big Data career path. The field is quite vast and it can be a very daunting task for anyone who starts learning Big Data. The technologies are numerous and it can be overwhelming to decide from where to begin.
In this post, we have summarised some of the jobs you can do under a Big Data career path. Each role requires different skills and responsibilities across a number of seniority levels.
A Data Scientist will be able to take a business problem and translate it to a data question, create predictive models to answer the question and storytell about the findings.
There are data scientists who fine-tune the statistical and mathematical models that are applied onto data. When somebody is applying their theoretical knowledge of statistics and algorithms to find the best way to solve a Data Science problem, they are filling the role of Data Scientist. When somebody builds a model to predict the number of credit card defaults in the next month, they are wearing the data scientist hat.
Data scientists are the bridge between the programming and implementation of Data Science, the theory of Data Science, and the business implications of data.
Knowledge of algorithms, statistics, mathematics, and broad knowledge of programming languages such as R and Python. Broad knowledge of how to structure a data problem, from framing the right questions to ask, to communicating the results effectively.
Rely mostly on their software engineering experience to handle large amounts of data at scale. These are versatile generalists who use computer science to help process large datasets. They typically focus on coding, cleaning up data sets, and implementing requests that come from data scientists. They typically know a broad variety of programming languages, from Python to Java. When somebody takes the predictive model from the data scientist and implements it in code, they are typically playing the role of a data engineer.
Focus on structuring the technology that manages data models and database administrators who focus on managing data storage solutions tend to be part of the category of data engineers.
Skills: A deep knowledge of data storage and warehousing solutions (SQL and NoSQL – based flavors), and programming frameworks such as Hadoop and Spark that can help you source data and process it.
Look through the data and provide reports and visualizations to explain what insights the data is hiding. When somebody helps people from across the company understand specific queries with charts, they are filling the data analyst role.
Business Analysts are a subset of data analysts that are more concerned with the business implications of the data and the actions that should result. Should the company invest more in project X or project Y? Business analysts will leverage the work of data science teams to communicate an answer.