Big Data Developer Data Ingestion circa £65K+, Jobs, 859

“Working with Lynne at SceneOne was an absolute pleasure.”

Barry Hunter (BSc PG) eHealth Engineer

curriculum vitae

/ˈviːtaɪ; ˈvaɪtiː/
noun (pl) curricula vitae
1. an outline of a person’s educational and professional history, usually prepared for job applications CV

Word Origin
Latin, literally: the course of one’s life

This vacancy is now closed

Big Data Developer - Data Ingestion - circa £65K+ - Southampton

Ref: 859 Date Posted: Wednesday 20 Feb 2019
LinkedIn ShareShare

SceneOne has an exciting opportunity to join of a multi-national, award-winning in IT Consulting with an aggressive growth plan for the UK market in 2018/19.  They service into both the private and public sector on crucial, cutting edge technology projects. 

Due to the continued success and growth of their UK operations and the onboarding of new and exciting clients, they are looking to add a skilled Big Data Developer, with strong experience of Data Ingestion, to join their Southampton team delivering a transformative cloud hosted data platform for a key customer.  

The Role

The ideal candidate will have a proven track record in implementing data ingestion and transformation pipelines for large scale organisations. They are seeking someone with deep technical skills in a variety of technologies to play an important role in developing and delivering early proofs of concept and production implementation.


  • Designing and implementing highly performant data ingestion pipelines from multiple sources using Apache Spark and/or Azure Databricks
  • Delivering and presenting proofs of concept to of key technology components to project stakeholders.
  • Developing scalable and re-usable frameworks for ingesting of geospatial data sets
  • Integrating the end to end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times
  • Working with event based / streaming technologies to ingest and process data
  • Working with other members of the project team to support delivery of additional project components (API interfaces, Search)



You will ideally have at least 3-5 years' experience in building solutions using a variety of open source tools a Microsoft Azure services and a proven track record in delivering high quality work to tight deadlines with experience and knowledge in the following areas:


  • Strong knowledge of Data Management principles
  • Experience in building ETL / data warehouse transformation processes
  • Direct experience of building data pipelines using Azure Data Factory and Apache Spark (preferably Databricks).
  • Experience using geospatial frameworks on Apache Spark and associated design and development patterns
  • Hands on experience designing and delivering solutions using the Azure Data Analytics platform (Cortana Intelligence Platform) including Azure Storage, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics
  • Experience with Apache Kafka / Nifi for use with streaming data / event-based data
  • Experience with other Open Source big data products Hadoop (incl. Hive, Pig, Impala)
  • Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB, Cassandra, Neo4J)
  • Experience working with structured and unstructured data including imaging & geospatial data.
  • Experience working in a Dev/Ops environment with tools such as Microsoft Visual Studio Team Services, Chef, Puppet or Terraform


The Package:

A competitive basic salary of circa £65K+ depending on experience, with extensive learning, development and progression opportunities and a generous benefits package. 

For more information please send your CV to or call 0141 899 0056.