Connecting to LinkedIn...


Big Data Engineer (Java/Scala/Hadoop) | New York

Job Title: Big Data Engineer (Java/Scala/Hadoop) | New York
Contract Type: Permanent
Location: New York
Salary: 200000
Start Date: ASAP
Contact Name: Jack Wooster
Contact Email:
Job Published: about 1 month ago

Job Description

Big Data Software Engineer (Java/Scala/Hadoop) | New York

We are searching for Java developers to join a start-up who are using cutting edge AI and Machine Learning in their platform to help provide their customers accurate real-time data to help influence their business decisions.


  • Have all round experience in developing and delivering large-scale business applications in scale-up systems as well as scale-out distributed systems.
  • Responsible for design and development of application on Cloudera or Hortonworks platform.
  • Should implement complex algorithms in a scalable fashion. Core data processing skills are highly important with SQL tools like HIVE/Impala/LLAP
  • Should be able to write MapReduce jobs or Spark jobs for implementation. Ability to write Java based middle layer orchestration between various components on Hadoop/spark stack. (example, taking a model definition in Java and scoring a dataset using Spark RDD on a cluster and storing output in a downstream application.
  • Work closely with product and Analytic managers, user interaction designers, and other software engineers to develop new product offerings and improve existing ones.


  • Core competency in Big Data and distributed systems, including data storage and management, searching, scheduling, logging, serialization, and modeling technologies

  • Proficiency in Core Java or Scala and Spark or Hadoop environments

  • Familiarity with dependency injection, Concurrency, Guice/Spring

  • Familiarity with JDBC API / Databases like MySQL, Oracle, Hadoop

  • Knowledge of graph databases(like OrientDb) and traversal

  • Experience in using file formats like XML, JSON, Avro, Parquet, Sequence, ORC and text files

  • Knowledge of SOLR/ElasticSearch, real-time data processing using Flume or Kafka or  Cloud based deployment would be preferred but not required.

  • Have experience in agile development methodologies, e.g. Scrum, Kanban.

Get in touch now to find out more about this opportunity. Salary $130,000 -$160,000


Big Data Software Engineer (Java/Scala/Hadoop) | New York