W1siziisimnvbxbpbgvkx3rozw1lx2fzc2v0cy9jbnrlbgxldgvjig5ldy9qcgcvbmv3lwjhbm5lci1kzwzhdwx0lmpwzyjdxq

Enterprise Data Engineer

  • Location

    New York

  • Sector:

    Data Science

  • Contact:

    Adam Oliver Brown

  • Contact email:

    adam@intelletec.com

  • Job ref:

    AOB260

  • Startdate:

    ASAP

  • Consultant:

    Adam Brown

We are partnered with one of America's most respected and revered investment firms as they look to add Data Engineers to their ranks in a number of business areas.

As a Data Engineer, you will be responsible for designing and building a quality cloud-based data pipeline and its peripheral tooling. You must be self-driven and comfortable working in a highly collaborative environment while interfacing with both technical and non-technical persons. In this role, you will join forces with other teams of software engineers and data scientists to support our business initiatives.  

 

Core Attributes: 

The ideal candidate will possess… 

·       design and analytical capabilities needed to build cloud-based data platforms 

·       collaboration and communication skills required to support our software engineering practices including: agile development, CI/CD, code review, etc. 

·       strong software engineering and computer science fundamentals along with software testing experience including: unit test, integration test, etc. 

·       a well-developed sense of ownership in their work, driving decisions from inception through completion 

·       passion for delivering value to the business with a relentless drive to improve process 

·       a keen desire to understand the domain knowledge around the products and how the products are developed along with knowledge of industry trends 

·       a strong sense of ethical responsibility as big data touches the privacy of our clients 

·       the ability to develop innovative solutions in a greenfield environment 

 

Principal Responsibilities: 

 

·       Create and maintain optimal data pipeline architecture 

·       Identify, design, and implement internal process improvements focused on: automating manual processes, optimizing data delivery, re-designing infrastructure for better scalability, etc. 

·       Build the infrastructure required for extraction, transformation, and loading of data from various data sources 

·       Build analytic tools to provide actionable insights into key business performance metrics 

·       Work with stakeholders across the business and technology to solve their data needs 

·       Be responsible for key aspects of project delivery at a high level of quality 

Experience in…  

·       building and optimizing “Big Data” data pipelines, architectures and data sets 

·       object oriented or functional languages (Java8+, Python, Scala, etc.) 

·       CI/CD Tools (Git, Docker) 

·       AWS cloud services (EC2, ECS, EMR, S3, RDS, etc.) 

·       ETL Orchestration Tools (Airflow, Glue, Luigi, NiFi, etc.) 

·       SQL and NoSQL data store (PostgreSQL, MySQL, DynamoDB, Elasticsearch, etc.) 

Knowledge of…  

·       big data tools and vendors (Hadoop, Spark, Kafka, Databricks, etc.) 

·       HashiCorp Tools (Terraform, Vault, Consul, etc.) 

·       big data concepts (message queuing, streaming processing, high availability, scalability, etc.) 

·       data security concepts (data governance, access control, etc.) 

·       the workflow of data wrangling, feature engineering, model development and model deployment along with necessary architectural requirements 

 

Soft Skills:  

·       Strong interpersonal skills and ability to forge relationships resulting in successful partnerships 

·       Ability to convey ideas effectively to help drive solutions for the business and tech partners 

·       Enthusiasm for learning and is results oriented

 

#LI-AB2