Job Description
We are searching for Hadoop developers to join a start-up who are helping provide accurate real-time data to help influence their business decisions using cutting-edge AI and Machine Learning in their new platform
You will predominantly be working with a team consisting of Engineers with expertise in Java/Scala/Hadoop but will have the chance to expand your skillset into other technologies and experiment with AI and ML techniques
Role:
- Have all round experience in developing and delivering large-scale business applications in scale-up systems as well as scale-out distributed systems.
- Responsible for design and development of application on Cloudera or Hortonworks platform.
- Should implement complex algorithms in a scalable fashion. Core data processing skills are highly important with SQL tools like HIVE/Impala/LLAP
- Should be able to write MapReduce jobs or Spark jobs for implementation. Ability to write Java-based middle layer orchestration between various components on Hadoop/Spark stack. (example, taking a model definition in Java and scoring a dataset using Spark RDD on a cluster and storing output in a downstream application.
- Work closely with product and Analytic managers, user interaction designers, and other software engineers to develop new product offerings and improve existing ones.
Requirements
- Core competency in Big Data and distributed systems, including data storage and management, searching, scheduling, logging, serialization, and modeling technologies
- Proficiency in Core Java or Scala and Spark or Hadoop environments
- Familiarity with dependency injection, Concurrency, Guice/Spring
- Familiarity with JDBC API / Databases like MySQL, Oracle, Hadoop
- Knowledge of graph databases(like OrientDb) and traversal
- Experience in using file formats like XML, JSON, Avro, Parquet, Sequence, ORC and text files
- Knowledge of SOLR/ElasticSearch, real-time data processing using Flume or Kafka or Cloud based deployment would be preferred but not required.
- Have experience in agile development methodologies, e.g. Scrum, Kanban.
If you are interested in this Hadoop Engineer role please apply directly to this advert.