Firmenlogo

Principal Data Engineer-R01570250 presso Brillio-2

Brillio-2 · Hyderabad, India · Hybrid

SQL

NoSQL

AWS

Cloud

Candidarsi ora

Description

Senior Tech Specialist, Applications & Platforms

Job requirements

Experience Range: 7 to 10 years of experience in advanced AWS data engineering, including significant expertise in Master Data Management (MDM) with Stibo and designing large-scale data solutions Key Responsibilities:
  • Design and architect scalable data pipelines and solutions using AWS native services to support complex business requirements
  • Develop, implement, and optimize ETL processes leveraging AWS technologies such as EMR, Redshift, Athena, and DMS
  • Configure and manage real-time data streaming and ingestion frameworks utilizing Kinesis and Amazon API Gateway
  • Monitor, troubleshoot, and enhance data infrastructure performance using CloudWatch and Open Search
  • Automate deployment and infrastructure provisioning with CloudFormation and manage schema migrations using SCT
  • Lead the implementation and integration of Master Data Management (MDM) solutions using Stibo within enterprise data platforms
  • Collaborate with cross-functional teams to integrate data solutions with enterprise applications and analytics platforms
  • Ensure data security, quality, and compliance by implementing best practices and leveraging AWS security features
  • Lead technical reviews, mentor junior engineers, and drive adoption of emerging data technologies within the organization
    • Required Skills:
  • Advanced proficiency in AWS services including SNS, SQS, Amazon API Gateway, Athena, CloudFormation, CloudWatch, DMS, DynamoDB, EMR, Kinesis, Open Search, Redshift, SCT
  • Expertise in designing and managing data pipelines and ETL workflows on AWS
  • Strong experience with Spark using Scala for large-scale data processing
  • Hands-on experience with Oozie workflow scheduling and orchestration
  • Deep knowledge of real-time data streaming with AWS Kinesis
  • Proven ability to configure and optimize Amazon Redshift for analytics workloads
  • Experience with DynamoDB for NoSQL database solutions
  • Ability to automate infrastructure and deployments using AWS CloudFormation
  • Proficient in monitoring and logging with AWS CloudWatch
  • Skill in managing data migration and schema transformation using AWS SCT and DMS
  • Demonstrated expertise in implementing and integrating MDM solutions using Stibo
    • Preferred Skills:
  • Experience integrating AWS data solutions with third-party analytics and visualization tools
  • Proficiency in optimizing performance for large-scale distributed data systems
  • Knowledge of emerging AWS data services and staying current with industry best practices
  • Experience with Open Search for search and analytics use cases
  • Background in mentoring and leading engineering teams in cloud data projects
  • Familiarity with advanced data governance and data quality frameworks in MDM environments
    • Desired Qualifications:
  • Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a closely related discipline
  • AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect – Professional
  • Certification in Apache Spark or relevant big data technologies (preferred)
    • Candidarsi ora

    Altri lavori

    Firmenlogo

    MEAN Stack Developer

    Distro · Noida,

    Firmenlogo

    Product Manager -Technical (Life Insurance & Annuity)

    Zinnia · Gurugram,

    Firmenlogo

    Senior Accountant-Revenue & Platform Accounting

    StockX · Bangalore,