We are working with an innovative, disruptive software house that provide business management solutions for thousands of clients, including major retail brands across Europe. This company are integrating machine learning into their software solutions and need a forward-thinking Senior Data Engineer to join a team of Data Scientists and Software Developers. You will be involved in the London Scrum team and there are various other scrum teams in Europe.

In this role you will…

• Create and maintain data pipelines to extract, transform and load data from various data sources.
• Work with DevOps to automate pipeline from/to production environments
• Support the data scientists to write scalable data processing and training jobs.

Expected skills & experiences…

• Python scripting.
• A successful history of manipulating, processing and extracting value from very large disconnected datasets (at least 100 millions of records / 100 TB).

Tech stack…

• Python / PyData stack.
• PySpark / Databricks.
• Azure Storage or S3.
• MongoDB or other No SQL database.
• Kafka – Bonus.
• Azure – Bonus.

Please apply below if you feel your skillset suits this position. Thank you for your interest.