We are partnered with one of America's most respected and revered investment firms as they look to add Data Engineers to their ranks in a number of business areas.
As a Data Engineer, you will be responsible for designing and building a quality cloud-based data pipeline and its peripheral tooling. You must be self-driven and comfortable working in a highly collaborative environment while interfacing with both technical and non-technical persons. In this role, you will join forces with other teams of software engineers and data scientists to support our business initiatives.
The ideal candidate will possess…
· design and analytical capabilities needed to build cloud-based data platforms
· collaboration and communication skills required to support our software engineering practices including: agile development, CI/CD, code review, etc.
· strong software engineering and computer science fundamentals along with software testing experience including: unit test, integration test, etc.
· a well-developed sense of ownership in their work, driving decisions from inception through completion
· passion for delivering value to the business with a relentless drive to improve process
· a keen desire to understand the domain knowledge around the products and how the products are developed along with knowledge of industry trends
· a strong sense of ethical responsibility as big data touches the privacy of our clients
· the ability to develop innovative solutions in a greenfield environment
· Create and maintain optimal data pipeline architecture
· Identify, design, and implement internal process improvements focused on: automating manual processes, optimizing data delivery, re-designing infrastructure for better scalability, etc.
· Build the infrastructure required for extraction, transformation, and loading of data from various data sources
· Build analytic tools to provide actionable insights into key business performance metrics
· Work with stakeholders across the business and technology to solve their data needs
· Be responsible for key aspects of project delivery at a high level of quality
· building and optimizing “Big Data” data pipelines, architectures and data sets
· object oriented or functional languages (Java8+, Python, Scala, etc.)
· CI/CD Tools (Git, Docker)
· AWS cloud services (EC2, ECS, EMR, S3, RDS, etc.)
· ETL Orchestration Tools (Airflow, Glue, Luigi, NiFi, etc.)
· SQL and NoSQL data store (PostgreSQL, MySQL, DynamoDB, Elasticsearch, etc.)
· big data tools and vendors (Hadoop, Spark, Kafka, Databricks, etc.)
· HashiCorp Tools (Terraform, Vault, Consul, etc.)
· big data concepts (message queuing, streaming processing, high availability, scalability, etc.)
· data security concepts (data governance, access control, etc.)
· the workflow of data wrangling, feature engineering, model development and model deployment along with necessary architectural requirements
· Strong interpersonal skills and ability to forge relationships resulting in successful partnerships
· Ability to convey ideas effectively to help drive solutions for the business and tech partners
· Enthusiasm for learning and is results oriented