Data Engineer II, Global Transportation Tech Services (GTTS) en Amazon

Description

Amazon moves millions of packages every day through the world's largest transportation network of planes, trucks, and delivery vehicles. Before any package can be assigned to a route, someone needs to design that network - figure out which lanes exist, how much capacity each node and lane can handle, and what changes are needed as demand shifts. That's what we build.

Global Transportation Tech Services (GTTS) creates software that manages the foundation of Amazon's outbound transportation network. Your work has direct financial impact on network efficiency and directly affects whether customers get their packages on time.

As a Data Engineer, you'll build the data infrastructure that powers our transportation systems. You'll design and implement data pipelines that process terabytes of transportation data daily, build data models that enable analytics and machine learning, and create the data foundations that drive operational decisions across Amazon's global network.

Key job responsibilities

• Working independently to own end-to-end delivery (from design through release) of data pipelines and medium-sized data projects

• Designing and implementing data models that support analytics, reporting, and ML use cases

• Working with AWS data technologies such as EMR, Glue, S3, Kinesis, Athena, Lake Formation and Redshift

• Building data pipelines using Spark + Scala/Python that process TBs of data per day

• Implementing data quality frameworks and monitoring for pipeline reliability

• Working with data consumers (analysts, scientists, business teams) to understand requirements and deliver solutions

• Contributing to design reviews and code reviews for data systems

• Participating in operational support for our data products by joining a regular on-call rotation

• Driving data quality, reliability, and process improvements

• Mentoring DE-I engineers and helping them grow technically

• Participating in regular hackathons to bring new ideas to GTTS

A day in the life

• You build and operate your data systems - we handle on-call for what we create

• Work spans multiple technologies: data pipelines, ETL/ELT, data warehousing, streaming, and integration with analytics and ML platforms

• Regular interaction with data consumers - analysts, scientists, and business stakeholders

• We value different perspectives and encourage open discussion

• Opportunity to work on data problems that affect transportation operations globally

• Team members work across different time zones

About the team

We're a global team with members in Hyderabad and Luxembourg. The team includes data engineers, software engineers, product managers, technical program managers and applied scientists working on transportation problems.

Basic Qualifications

• 3+ years of data engineering experience

• Experience with SQL

• Experience with data modeling, warehousing and building ETL pipelines

• Experience with big data technologies such as: Hadoop, Hive, Spark, EMR

• Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS

• Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets

Preferred Qualifications

• Knowledge of professional software engineering & best practices for full software development life cycle, including coding standards, software architectures, code reviews, source control management, continuous deployments, testing, and operational excellence

• Experience providing technical leadership and mentoring other engineers for best practices on data engineering

• Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions

• Knowledge of batch and streaming data architectures like Kafka, Kinesis, Flink, Storm, Beam

• Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases)

• Experience with Redshift, Oracle, NoSQL etc.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.