Company:
We are closely working with a distinguished name in the realm of consumer credit ratings. Along with consumer credit scores, they are also known for their unwavering commitment to provide credit opinions that are independent, forward-looking, and deeply insightful.
Key Responsibilities:
- Constructing data pipelines and applications to seamlessly stream and process data sets at high speed.
- Architecting the necessary infrastructure for optimal data extraction, transformation, and loading from various data sources using SQL, NoSQL, Kafka, and AWS Big Data technologies.
- Teaming up with Data Product Managers, Data Architects, and fellow Data Engineers to conceptualize, develop, and deliver impactful data solutions.
- Creating analytics tools that leverage data pipelines to provide actionable insights into customer acquisition, operational efficiency, and other vital business performance metrics.
- Monitoring data lineage, enhancing data quality, and promoting data discoverability.
- Actively participating in an Agile (Scrum) environment and collaborating with cross-functional teams (Product Owners, Scrum Masters, Developers, Designers, Data Analysts).
Qualifications & Skills:
- Profound experience in Java development.
- A minimum of 5 years in a data engineering role, ideally with a focus on developing large data pipelines.
- Robust SQL and NoSQL skills with the capability to generate queries for data extraction and build performant datasets.
- Familiarity with relational SQL and NoSQL databases, including RDBMS (Oracle, Postgres) and NoSQL systems (Cassandra, Mongo, or Redis).
- Practical experience with distributed systems like Spark, Hadoop (HDFS, Hive, Presto, PySpark) for data querying and processing.
- Hands-on experience with message queuing and stream data processing systems (like Kafka Streams).
- Sharp analytical skills, especially in dealing with unstructured datasets.
- Proficiency in AWS cloud services: EC2, Lambda, S3, Athena, Glue, and EMR.
