Company Overview

Incedo is a US-based consulting, data science and technology services firm with over 3000 people helping clients
from our six offices across US, Mexico and India. We help our clients achieve competitive advantage through
end-to-end digital transformation. Our uniqueness lies in bringing together strong engineering, data science, and
design capabilities coupled with deep domain understanding. We combine services and products to maximize
business impact for our clients in telecom, Banking, Wealth Management, product engineering and life science
& healthcare industries. 
Working at Incedo will provide you an opportunity to work with industry leading client organizations, deep
technology and domain experts, and global teams. Incedo University, our learning platform, provides ample
learning opportunities starting with a structured onboarding program and carrying throughout various stages of
your career. A variety of fun activities is also an integral part of our friendly work environment. Our flexible
career paths allow you to grow into a program manager, a technical architect or a domain expert based on your
skills and interests. 
Our Mission is to enable our clients to maximize business impact from technology by

• Harnessing the transformational impact of emerging technologies
• Bridging the gap between business and technology

Role Description

• Work with the customer, users, technical architects, and application designers to define the data requirements and structure for the application.
• Model and design the application data structure, storage, and integration.
• Lead the database analysis, design, and build effort.
• Work with the application architects and designers to design the integration solution.
• Ensure that the database designs fulfill the requirements, including data volume, frequency needs, and long-term data growth.
• Understand Business Requirements and help in developing High level and Low-level Data Engineering and Data Processing Documentation
• Designing Solution architecture, work on Data Ingestion, Data Preparation, Data Transformation, Data Load
• Develop Data Processing Jobs (that focus on Data Engineering tasks like Data Massaging, Data Cleansing, Data Transformation – both ETL and ELT along with load into Data Base)
• Strong hands-on experience of programming languages like Python, Scala, and Java.
• Knowledge of how distributed systems work is essential for understanding the architecture and design principles of Apache Kafka.
• Familiarity with messaging systems and concepts such as pub/sub, message queues, and event-driven architecture is beneficial for learning Kafka.
• Knowledge of big data technologies such as Hadoop, Spark, and HBase can clarify the use cases and integrations with ELK, Exadata and Apache Kafka.
• Understanding real-time data processing concepts and technologies like Apache Storm, Apache Flink, or Spark Streaming can be valuable for using Kafka in real-time data pipelines.
• Having experience working with Elasticsearch, Logstash and Kibana.
• Knowledge of cloud platforms such as AWS, can be beneficial for deploying and managing Kafka clusters in the cloud.
• Hands on working Experience with AWS Services like EMR, Kinesis, S3, Cloud Formation, Glue, API Gateway, Lake Foundation, AWS SageMaker.
• Feature Engineering/Data Processing to be used for Model development.
• Experience building data pipelines for structured/unstructured, real-time/batch, events/synchronous/ asynchronous using MQ, Kafka, Steam processing.
• Knowledge of developing efficient frameworks for development and testing using (Sqoop/Nifi/Kafka/Spark/Streaming/ WebHDFS/Python) to enable seamless data ingestion processes on to the Hadoop platform.
• Enabling Data Governance and Data Discovery.
• Exposure of Job Monitoring framework along validations automation
• Exposure of handling structured, Un Structured and Streaming data.

Technical Skills

• Experience with building data platform on cloud (Data Lake, Data Warehouse environment, Databricks)
• Strong technical understanding of data modeling, design and architecture principles and techniques across master data, transaction data and derived/analytic data
• Proven background of designing and implementing architectural solutions which solve strategic and tactical business needs.
• Deep knowledge of best practices through relevant experience across data-related disciplines and technologies, particularly for enterprise-wide data architectures, data management, data governance and data warehousing Highly competent with database design
• Highly competent with data modeling
• Strong Data Warehousing and Business Intelligence skills or including Handling ELT and scalability issues for enterprise level data warehouse.
• Creating ETLs/ELTs to handle data from various data sources and various formats.
• Solid hands-on and Solution Architecting experience in Cloud Technologies – AWS, Azure and GCP.
• Hands-on experience on Data Ingestion using Apache Nifi, Apache Airflow, Sqoop, and Ozzie
• Hands on working experience of data processing at scale with event driven systems, message queues (Kafka/ Flink/Spark Streaming)
• Hands on working Experience with AWS Athena
• Experience gathering and processing raw data at scale (including writing scripts, web scraping, calling APIs, write SQL queries, etc.)
• Hands-on working experience in analyzing source system data and data flows, working with structured and unstructured data.
• Must be very strong in writing Spark SQL queries.
• Strong organizational skills, with the ability to work autonomously as well as in a team-based environment.
• Pleasant Personality, Strong Communication & Interpersonal Skills

Nice-to-have skills

• Stores - HIVE, ELK, Oracle Exadata, Mongo DB, Kafka
• Orchestration – MQ, Kafka, Airflow
• Streaming – Kafka
• Cloud – AWS
• Bash and Scripting: Unix or Shell scripting
• Exposure to various ETL and Business Intelligence tools
• Experience in data warehouse design and best practices.
• A strong background in troubleshooting and technology support will be beneficial.

Qualifications

• A bachelor's degree in computer science, computer engineering, or a related discipline is required to work as a technical lead.
• Certification in Spark Databricks and AWS would be a big plus.
• Individuals in this field can further display their leadership skills by completing the Project Management Professional certification offered by the Project Management Institute.
• Confluent Certification for Apache Kafka
• Hadoop Certified Data Engineer  
• AWS Solution Architect Certification

Company Value

We value diversity at Incedo. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.