We are looking for an experienced Lead Data Engineer to join our team in Reading / London!
The key focus of this role is creating, maintaining and evolving a key set of feed applications which improve quality and quantity of the numerous (>50) “consumer panels” which Kantar operates around the globe.
A consumer panel is a group of users who have volunteered to answer online surveys. These users are “opted in” with all relevant permissions already provided, and have filled out demographic questionnaires. This allows Kantar to send them relevant online surveys.
In recent years, the Market Research industry has been an increasing target of online fraud. This means that in some markets where surveys are incentivised (typically economically eg. small payments per survey completed which may be converted to gift cards etc), fraudsters are targeting survey panels and completing surveys to obtain financial incentives. As with any sophisticated and scaled fraud attacks, the attacks are constantly evolving.
At Kantar, we need to protect against these kinds of fraud attacks (both existing patterns and novel attacks) and constantly ensure that we are able to detect and remove any fraudulent panellists. We already have a range of methods in production to provide protection, however, we believe that we need to stay ahead of the evolving fraud and invest in this area so we can always provide the best data and the best answers for our customers globally.
We know that better decisions can be made when the right data and more timely and relevant information is available to decisioning tools. Our sophisticated analytics, Data Science and AI models need data delivered to them in a reliable and low-latency manner. Additionally, we need new data pipelines to bring new datasets to the Data Science team so we can evolve and improve existing models, or create new ones.
There is significant potential for the right person to make a real difference in this area and work directly with Operational, Engineering and Data Science leaders.
We are looking for an Lead Data Engineer who will work within an existing team of expert Data Analysts/Scientists/Engineers and a team of experienced and talented software developers. Prior experience of working with similar topics in an operational environment is preferred but not necessary.
You will be developing your skill levels and working with the latest technologies as you progress with the company.
What You'll Be Doing:
Working with the existing Data Science team, you will produce new dashboards, reports and recommendations to help maintain Kantar’s ability to offer the best panellists, and refine Kantar’s ability to measure and act on panellist quality.
Be a key stakeholder on our cloud based infrastructure, providing expertise and recommendations on the best opportunities to innovate and develop the underlying technology
Integrate data from multiple sources, such as databases, APIs, and external data sets, into a unified and accessible data ecosystem, to support applications, data science and operational reporting
Maintain comprehensive documentation of data pipelines, data models, and data related processes to facilitate collaboration and knowledge sharing.
Take primary responsibility to design, develop, and maintain data pipelines to extract, transform, and load data from various sources, ensuring data accuracy and reliability.
Proactively review existing process to identify opportunities to automate manual processes, optimise data delivery, re-design infrastructure for greater scalability
Set up monitoring systems and alerts to proactively detect and address issues in data pipelines or infrastructure, minimising downtime, and data loss.
Collaborating with data scientists, analysts and data visualisation specialists to identify potential opportunities and maximise the value delivered from data models
Provide guidance and mentorship to data engineers, helping them develop their skills and grow in their roles.
Communicate and present information to non-technical team members including senior Commercial stakeholders
What You'll Bring:
5+ years of experience in a Data Engineering role
Experience as a data engineer working in with experience in key disciplines, e.g. data warehousing, Business Intelligence and big data / data science
Experience working with AWS (preferred, but other cloud experience e.g. Azure is also relevant).
Experience in developing and maintaining ELT/ETL data pipelines.
Ability to translate business needs into technical specifications
Strong knowledge of Python and/or R
Strong knowledge of SQL
Proficient with different data modelling techniques.
Proficient with CI/CD principles and tools
Some experience of Business Intelligence (BI) tools such as PowerBI
Experience with DBT(Database Build Tool) desirable.
Strong problem-solving skills with an emphasis on agile / iterative product development along with logical and critical thinking
Familiarity with Agile methodologies or experience working in a fast-paced project environment
Diligent and attentive to detail, have a strong work ethic and ability to collaborate with different teams in the company
Able to contribute to improving software quality across the Data Science and Software Engineering teams
Our Tech Stacks:
AWS
Redshift
Postgres
DBT
PowerBI
Current environment is AWS, but most of the rest of the company is Azure, and that is the direction of travel for new development.
Is this you? Send us your application today!
Please note that candidates will require the right to work in the UK.