Big Data Engineer - Java/Scala

We are searching for Java developers to join a start-up who are helping provide accurate real-time data to help influence their business decisions using cutting-edge AI and Machine Learning in their new platform

You will predominantly be working with a team consisting of Engineers with expertise in Java/Scala/Hadoop but will have the chance to expand your skillset into other technologies and experiment with AI and ML techniques


  • Have all round experience in developing and delivering large-scale business applications in scale-up systems as well as scale-out distributed systems.
  • Responsible for design and development of application on Cloudera or Hortonworks platform.
  • Should implement complex algorithms in a scalable fashion. Core data processing skills are highly important with SQL tools like HIVE/Impala/LLAP
  • Should be able to write MapReduce jobs or Spark jobs for implementation. Ability to write Java-based middle layer orchestration between various components on Hadoop/Spark stack. (example, taking a model definition in Java and scoring a dataset using Spark RDD on a cluster and storing output in a downstream application.
  • Work closely with product and Analytic managers, user interaction designers, and other software engineers to develop new product offerings and improve existing ones.


  • Core competency in Big Data and distributed systems, including data storage and management, searching, scheduling, logging, serialization, and modeling technologies
  • Proficiency in Core Java or Scala and Spark or Hadoop environments
  • Familiarity with dependency injection, Concurrency, Guice/Spring
  • Familiarity with JDBC API / Databases like MySQL, Oracle, Hadoop
  • Knowledge of graph databases(like OrientDb) and traversal
  • Experience in using file formats like XML, JSON, Avro, Parquet, Sequence, ORC and text files
  • Knowledge of SOLR/ElasticSearch, real-time data processing using Flume or Kafka or Cloud based deployment would be preferred but not required.
  • Have experience in agile development methodologies, e.g. Scrum, Kanban.


If you are interested in this Big Data Engineer role please apply directly to this advert.