- Senior
- Escritório em Chennai
Job Title: AWS Data Engineer (IoT/Streaming Data)
📍 Location: Chennai or Bangalore.
💼 Experience: 3+ Years
🕒 Employment Type: Full-time
💼 Experience: 3+ Years
🕒 Employment Type: Full-time
About the Role
We are seeking a highly skilled AWS Data Engineer to design, build, and optimize large-scale data pipelines and analytics platforms. The ideal candidate will have strong hands-on expertise in AWS Glue, Athena, Redshift, PySpark, and SQL, with a solid understanding of data engineering and data warehousing concepts. You will work on batch and real-time data streaming solutions to enable data-driven decision-making across the organization.
What You Will Do
- Design and implement streaming data pipelines from IoT sensors, cameras, vehicle telemetry, and infrastructure systems.
- Build scalable infrastructure using AWS Kinesis, Apache Flink, and Spark for real-time and batch workloads.
- Enable time-series feature stores and sliding window processing for mobility and telemetry patterns.
- Integrate simulation outputs and model predictions into AWS data lakes.
- Maintain data validation, schema versioning, and high-throughput ingestion pipelines.
- Collaborate with Data Scientists and Simulation Engineers to optimize data formats (e.g., Parquet, Protobuf, Delta Lake).
- Deploy, monitor, and optimize pipelines on AWS cloud and/or edge infrastructure.
You Are a Strong Candidate If You Have
- 3+ years of experience in Data Engineering, preferably with real-time streaming systems.
- Proficiency in Python, SQL, and distributed data systems (Kinesis, Spark, Flink, etc.).
- Strong understanding of event-driven architectures, data lakes, and message serialization.
- Experience with sensor data processing, telemetry ingestion, or mobility data is a plus.
- Familiarity with Docker, CI/CD, Kubernetes, and cloud-native architectures.
- Experience building workflows and orchestrating pipelines (e.g., Airflow, Step Functions, Dagster).
Preferred Qualifications
- Exposure to smart city platforms, V2X ecosystems, or other time-series paradigms.
- Experience integrating data from cameras and IoT sensors into real-time or batch pipelines.
