W1siziisimnvbxbpbgvkx3rozw1lx2fzc2v0cy9jbnrlbgxldgvjig5ldy9qcgcvbmv3lwjhbm5lci1kzwzhdwx0lmpwzyjdxq

Hadoop Developer - AI/ML Platform

  • Location

    New York

  • Sector:

    Software Engineering

  • Job type:

    Permanent

  • Salary:

    $250,000

  • Contact:

    Joe Hunt

  • Contact email:

    joe@intelletec.com

  • Job ref:

    JHunt - TwHAD

  • Startdate:

    ASAP

We are searching for Hadoop developers to join a start-up who are helping provide accurate real-time data to help influence their business decisions using cutting-edge AI and Machine Learning in their new platform

You will predominantly be working with a team consisting of Engineers with expertise in Java/Scala/Hadoop but will have the chance to expand your skillset into other technologies and experiment with AI and ML techniques

Role:

  • Have all round experience in developing and delivering large-scale business applications in scale-up systems as well as scale-out distributed systems.
  • Responsible for design and development of application on Cloudera or Hortonworks platform.
  • Should implement complex algorithms in a scalable fashion. Core data processing skills are highly important with SQL tools like HIVE/Impala/LLAP
  • Should be able to write MapReduce jobs or Spark jobs for implementation. Ability to write Java-based middle layer orchestration between various components on Hadoop/Spark stack. (example, taking a model definition in Java and scoring a dataset using Spark RDD on a cluster and storing output in a downstream application.
  • Work closely with product and Analytic managers, user interaction designers, and other software engineers to develop new product offerings and improve existing ones.

Requirements

  • Core competency in Big Data and distributed systems, including data storage and management, searching, scheduling, logging, serialization, and modeling technologies
  • Proficiency in Core Java or Scala and Spark or Hadoop environments
  • Familiarity with dependency injection, Concurrency, Guice/Spring
  • Familiarity with JDBC API / Databases like MySQL, Oracle, Hadoop
  • Knowledge of graph databases(like OrientDb) and traversal
  • Experience in using file formats like XML, JSON, Avro, Parquet, Sequence, ORC and text files
  • Knowledge of SOLR/ElasticSearch, real-time data processing using Flume or Kafka or Cloud based deployment would be preferred but not required.
  • Have experience in agile development methodologies, e.g. Scrum, Kanban.

If you are interested in this Hadoop Engineer role please apply directly to this advert.