In this role, you will:

📍Participate in architecture and engineering decisions, bringing your strong experience and knowledge to bear. 📍Ensure the security, integrity, and compliance of data according to industry and company standards. 📍Ensure our analytics and data platforms can scale reliably to the next several orders of magnitude 📍Accelerate company productivity by empowering your fellow engineers, researchers, and teammates with excellent data tooling and systems, providing a best in case experience 📍Bring new features and capabilities to the world by partnering with product engineers, trust & safety and other teams to build the technical foundations 📍Like all other teams, we are responsible for the reliability of the systems we build. This includes an on-call rotation to respond to critical incidents as needed

You might thrive in this role if you have:

📍Experience in building stream and batch data processing pipelines, using technologies such as, or equivalent to, Kafka, Spark, Flink. 📍Proficient with modern infrastructure management tools, such as Kubernetes and Terraform. 📍Have a passion for observability systems (bonus if for ML training). You are excited by the idea of building bespoke analytics systems that provide answers to key ML research questions. 📍Have worked in a ML training organization and have experience with the problem of data transformation during pre-training. 📍Are a proficient software engineer, ideally in Python, and are used to working with large monorepo codebases. 📍Have worked on data lifecycle management systems for large organizations, and dealt with the problems of access control, provenance, auditing, data movement at scale, metadata management, etc. 📍Are a self-starter and are comfortable operating in a fast-paced environment.