Data Engineer en Bespoke Technologies, Inc.
Bespoke Technologies, Inc. · Dulles, Estados Unidos De América · Onsite
- Senior
- Oficina en Dulles
BT-134 – Data Engineer
Skill Level: Principal I – ESW05
Location: Dulles (Hybrid) (fully on-site, no remote option)
**MUST HAVE A SECRET SECURITY CLEARANCE TO APPLY. Those who do not have a Secret clearance (or higher) will not be considered.**
Bespoke Technologies is seeking a Senior Data Engineer working to support the migration of applications, services and platforms to a Medallion Model.
Responsibilities:
- Data Pipeline Developer: design, develop and implement, end-to-end data ingestion and processing pipelines using various Cloud and Platform Technologies.
- Transform incoming bronze data (RAW) into silver (clean) and finally projections into gold tier. (Medallion model)
- Schedule, orchestrate and monitor automated/semi-automated data pipelines.
- Perform mapping and integration of data assets in support of the execution of multiple clients.
- Develop and maintain the data engineering team process improvement plan.
- Develop code using various programming and scripting languages to automate data ingestion, pipeline orchestration and improve data management processes.
- Ingest transparency, leveraging technologies such as AWS CloudWatch to identify places for measuring and gathering performance information on automated data pipelines.
- Ensure Data Engineering Team Standard Operating Procedures are appropriately captured and communicated across the team.
- Ensure technical correctness, timeliness and quality of delivery for the team.
- Demonstrate excellent oral and written communication skills to all levels of management.
Required Skills:
- Active Secret (S) clearance.
- 8+ years of software engineering experience.
- Minimum of 4 years of experience performing data engineering work in a cloud environment.
- Experience with relational, noSQL and/or file-based storage (e.g. Databricks, Elastic, Postgres, etc.)
- Experience with DevOps tools such as Jenkins, Docker, Ansible, Puppet, Chef, Splunk, etc.
- Working knowledge of Databricks, Cloud Relational Database Services and Elasticsearch
Desired Skills:
- Experience with Databricks workflows, Databricks Unity Catalog, Databricks Autoloader, and Databricks Delta Live Tables and Apache Spark.
- Basic working knowledge with Apache Niagara Files (NiFi)
- Experience with collaboration tools including MS Teams, MS Outlook, MS SharePoint, and Confluence
- Amazon Web Services (AWS) Professional certification or equivalent.
Desired Certifications:
- DoD 8570.1-M Compliance at IAT Level I (e.g., Certified Information Systems Security Professional (CISSP)) certification highly desired
- Bachelor’s degree in Software Engineering, Computer Science or a related discipline is required. [Ten (10) years of experience (for a total of eighteen (18) or more years) may be substituted for a degree.]
