Senior Data Engineer

Role Overview

We are seeking a highly skilled Senior Data Engineer to join the Data Products Delivery team. The ideal candidate will be responsible for designing, developing, and maintaining scalable, secure, and high-performing data pipelines that support Business Intelligence, Analytics, and AI/ML initiatives.

This role requires strong expertise in Python, PySpark, SQL, Databricks, and API-driven integrations, along with hands-on experience building enterprise-grade data solutions using Lakehouse and Medallion Architecture principles.

The candidate will work closely with Data Architects, Data Scientists, BI Developers, and CloudOps teams in a release-based Agile environment to deliver reliable and reusable data products.

Required Qualifications

Technical Skills

* Strong hands-on experience with:

* Python

* PySpark / Apache Spark

* Advanced SQL

* Extensive experience with Databricks, Delta Lake, and Lakehouse Architecture.

* Proven experience implementing Medallion Architecture in enterprise environments.

* Strong expertise in API-driven data ingestion (REST APIs, JSON, OAuth, pagination, throttling).

* Experience working with structured, semi-structured, and streaming data sources.

Cloud & DevOps

* Experience with cloud platforms such as AWS, Azure, or GCP.

* Familiarity with CI/CD pipelines and Infrastructure as Code concepts.

* Knowledge of secure authentication, secrets management, and DevSecOps practices.

* Understanding of cost optimization and FinOps principles.

Data & Analytics

* Experience supporting enterprise BI and Analytics workloads.

* Strong understanding of data modeling and reporting structures.

* Experience enabling data consumption for AI/ML and advanced analytics use cases.

Agile Delivery

* Experience working within Agile Scrum or Scrum of Scrums environments.

* Comfortable supporting release-based delivery cycles and production operations.

Nice to Have

* Experience exposing data through internal or external APIs.

* Familiarity with streaming technologies such as Kafka, Kinesis, Firehose, or Event Hubs.

* Experience with data governance, lineage, and cataloging tools.

* Exposure to feature stores and ML data pipelines.