Platzhalter Bild

Senior Data Engineer ( Azure ) bei Egen

Egen · Hyderabad, Indien · Hybrid

Python

SQL

DevOps Automation Tools

Microsoft Excel

Cloud

Jetzt bewerben
Job Overview:

We are seeking a highly skilled Senior Data Engineer with deep expertise in Microsoft Azure data ecosystem to design, develop, and maintain scalable data pipelines and architectures. The ideal candidate will play a key role in building robust data solutions that support advanced analytics, BI, and AI workloads across the organization. This role involves working with cross-functional teams — data scientists, analysts, and business stakeholders — to ensure high-quality, secure, and performant data delivery.

Total experience :  8 to 10 years

Key Responsibilities:


Job Overview:We are seeking a highly skilled Senior Data Engineer with deep expertise in Microsoft Azure data ecosystem to design, develop, and maintain scalable data pipelines and architectures. The ideal candidate will play a key role in building robust data solutions that support advanced analytics, BI, and AI workloads across the organization. This role involves working with cross-functional teams — data scientists, analysts, and business stakeholders — to ensure high-quality, secure, and performant data delivery.Total experience :  8 to 10 years Key Responsibilities:

1. Data Architecture & Pipeline Development
  • Design, develop, and maintain data ingestion, transformation, and storage pipelines using Azure Data Factory, Azure Databricks, and Synapse Analytics.
  • Build end-to-end ETL/ELT workflows from diverse sources such as APIs, on-prem databases, SaaS applications, and data lakes.
  • Implement data modeling (star schema, snowflake, data vault, etc.) for analytical and operational data stores.
  • Manage data ingestion frameworks for both batch and streaming (real-time) use cases using Azure Event Hubs / Azure Stream Analytics / Kafka.


    • 2. Data Management & Governance
  • Ensure data quality, consistency, and integrity across all environments.
  • Implement and enforce data governance standards, including metadata management, lineage tracking, and data cataloging (e.g., Azure Purview).
  • Manage security, compliance, and access controls using Azure Active Directory (AAD) and RBAC principles.
  • Automate data validation, auditing, and monitoring workflows using modern DevOps practices.


    • 3. Cloud & DevOps Integration
  • Manage and optimize Azure Data Lake Storage (ADLS Gen2) for cost and performance.
  • Leverage Infrastructure as Code (IaC) tools such as Terraform, Bicep, or ARM templates for environment provisioning.
  • Implement CI/CD pipelines for data workflows using Azure DevOps / GitHub Actions.
  • Optimize compute and storage costs while maintaining scalability and resilience.


    • 4. Collaboration & Leadership
  • Partner with data scientists, BI developers, and product teams to deliver reliable data solutions.
  • Mentor junior data engineers on best practices, coding standards, and Azure services.
  • Participate in architectural reviews and contribute to data platform design decisions.
  • Communicate technical insights and trade-offs to non-technical stakeholders effectively.



    • Core Azure Services:
  • Azure Data Factory (ADF) – pipeline orchestration, triggers, linked services, integration runtime.
  • Azure Databricks – PySpark, Delta Lake, notebooks, job orchestration.
  • Azure Synapse Analytics – dedicated and serverless SQL pools, data modeling, query optimization.
  • Azure Data Lake Storage (ADLS Gen2) – hierarchical namespace, lifecycle management.
  • Azure Functions / Logic Apps – event-driven data workflows.
  • Azure Event Hubs / Kafka / Stream Analytics – real-time data ingestion.


    • Programming & Tools:
  • Strong in Python or Scala for data transformations.
  • Proficient in SQL (T-SQL, Spark SQL, or Synapse SQL).
  • Experience with PySpark for distributed data processing.
  • Knowledge of Git, CI/CD, and DevOps best practices.
  • Familiarity with Power BI integration or other reporting tools is a plus.


    • Data Architecture & Modeling:
  • Expertise in data warehouse design, data lakehouse architecture, and data pipelines.
  • Understanding of data partitioning, indexing, and query optimization techniques.
  • Experience with dimensional modeling, slowly changing dimensions, and fact tables.


    • Other Skills:
  • Experience with API-based data ingestion and RESTful integrations.
  • Exposure to machine learning data pipelines or feature stores is a plus.
  • Familiarity with cost optimization and performance tuning in Azure.


    • Soft Skills
  • Excellent analytical and problem-solving abilities.
  • Strong communication and documentation skills.
  • Ability to work in an agile, fast-paced environment.
  • Proven experience collaborating in cross-functional teams.
  • Self-driven with a passion for data engineering and cloud technologies.
    • Jetzt bewerben

    Weitere Jobs

    Firmenlogo

    Senior Software Engineer in Test

    Guardant Health · Hyderabad,

    Firmenlogo

    Senior Azure/M365 Engineer

    McAfee · Bengaluru,

    Firmenlogo

    Solutions Engineer (Google Workspace)

    Zuora · Chennai,