Senior Data Specialist
Primary SkillsETL Fundamentals, SQL, BigQuery, Dataproc, SQL (Basic + Advanced), Python, Data Catalog, Data Warehousing, Composer, Dataflow, Cloud Trace, Cloud Logging, Cloud Storage, Datafusion, Modern Data Platform Fundamentals, Data Modelling Fundamentals, PLSQL, T-SQL, Stored Procedures Job requirementsAzure Data Engineer
Strong with PySpark (esp. Spark SQL) for complex transformation pipelinesHands-on with Airflow for orchestration and BigQuery SQL for querying and data modelingGood experience in GCP (BigQuery, GCS, Pub/Sub, Cloud Run, Functions,Cloud SQL)Comfortable with real-time ingestion: Kafka, webhooks, file-based triggersSolid API integration skills (REST/webhooks), with ability to handle payload-driven workflowsExperience working in Kubernetes (GKE) for deploying and scaling pipelinesComfortable handling UI-driven configuration, YAML-based setups, and modular frameworksExposure to schema transformation, data validation, hashing, and DQF logicDomain familiarity with logistics/healthcare (CVS context is a big plus)Strong ownership mindset — able to work under pressure and balance speed with reliabilityBasic understanding of BASH, SFTP transfers, Networking, Access Management Bonus: PostgreSQL, CI/CD, monitoring, dashboarding, or lightweight UI development=================================Python & PySpark (Spark SQL) – 3+ yearsAirflow (or any orchestration tool) – 2+ yearsGoogle Cloud Platform (BigQuery, GCS, Pub/Sub, Cloud Run, Functions, Cloud SQL) – 3+ yearsReal-time data ingestion (Kafka, webhooks, file-based) – 2+ yearsAPI integration (REST/webhooks) – 2+ yearsKubernetes (GKE preferred) – 1–2 yearsBigQuery SQL & PostgreSQL – 2+ yearsYAML/config-driven pipeline design – 2+ yearsSchema transformation, hashing, DQF – 2+ yearsCI/CD, observability, lightweight dashboards (Grafana/Streamlit/Flask UI) – 1+ yearLogistics or healthcare domain exposure – nice to have
Know more about:
Equal Employment Opportunity Declaration
Brillio is an equal opportunity employer to all, regardless of age, ancestry, colour, disability (mental and physical), exercising the right to family care and medical leave, gender, gender expression, gender identity, genetic information, marital status, medical condition, military or veteran status, national origin, political affiliation, race, religious creed, sex (includes pregnancy, childbirth, breastfeeding, and related medical conditions), and sexual orientation.
#LI-SR1