Connecting to LinkedIn...

Senior BigData Engineer (Python)

Job Title: Senior BigData Engineer (Python)
Contract Type: Permanent
Location: San Francisco
Salary: Circa $180,000- $250,000
Contact Name: Jason Rumney
Contact Email:
Job Published: almost 2 years ago

Job Description

My client a leading provider of a mobile first data platform who helps to collect and analyses behavioural data from over half a billion mobile users globally.

Due to expansion, my client is seeking a Senior Data Engineer to join a developing team to support a data pipeline team developing scalable machine learning algorithms, data acquisition and data visualisation.


  • Work closely with the Data Science team to develop scalable machine learning algorithms to understand user’s behaviour in production environment
  • Develop real-time analytics data pipeline processing billions of events
  • Work with QA and DevOps teams to troubleshoot any issues that may arise during production
  • Data acquisition, data gathering from different sources for example US census data, mobile application data, movies and music data, location data like Points of Interest,
  • Benchmark and evaluate new Big Data technologies for later implementation

Desirable experience:

  • 4+ years of experience in software development in Java or Python
  • Big Data technologies familiarity with Hadoop, Storm or Spark
  • Ability to use one or more of these technologies: ElasticSearch, Aerospike, Kafka, SQL, NoSQL
  • Familiar with AWS environment
  • Functional programming and object oriented programming
  • Degree in software engineering, data science, statistics, data mining, or similar field  

Optional experience:

  • Scala, JavaScript, REST APIs, AWS
  • Web development (AngularJS or NodeJS)
  • IoT (Internet of Things)
  • Virtualization technologies like Docker, Vagrant or Mesosphere

Our Benefits:

  • Competitive salary
  • 100% of premiums for medical, dental, vision and life insurance – Platinum plans
  • 401k plan
  • Balanced work and life schedule and remote work options
  • Paid vacation time, holidays and sick days
  • Catered food and snacks
  • Pet-friendly work space

Note that salary on offer is up to $190k,to find out more please email