Job Description
My client is a very well-funded, Series-C global streaming platform headquartered in San Francisco with offices in LA, NYC, and Beijing - They are the fastest growing LEGAL free streaming platform in the world.
I am looking for a Senior Data Engineer:
Responsibilities:
- Design, implement and build ETL pipelines that deliver data with measurable quality
- Understand the data needs, including schema, the frequency of the data by interacting with machine learning engineers and data scientists
- Develop Scala (or equivalent) scripts to generate enriched tables or logs using distributed scalable platforms (Databricks)
What's Required:
- 5+ years of a proven track record in a data engineering or related role.
- Ability to take ownership of the design, implementation, and maintenance of scalable end to end data pipelines.
- Ability to quickly evaluate and make trade-off decisions on adopting emerging technologies.
- Strong knowledge of Scala and Apache Spark 2.0+.
- Building real-time data pipelines using Redshift, S3, Kinesis, Spark structured streaming, Akka streams, and similar stacks on leading cloud platforms.
- Experience with DAG workflow schedulers like Airflow.
- A passion for shipping production quality code with good test coverage using testing frameworks for testing Spark code
- BSc/MSc in Computer Science or related field
On Offer:
- Base salary up to $250k
- Equity
- Amazing benefits - Unlimited PTO, catered lunches etc
