Job Opportunities API

Position overview Digitribe is looking for a Data Engineer with deep expertise in Azure and Databricks to design, develop, and maintain scalable data engineering solutions. You will build production-grade ETL/ELT pipelines, develop data processing applications, and contribute to a modern lakehouse architecture. Working cross-functionally with Architecture, Data, AI, Cloud, and CI/CD teams, you will translate business and analytical requirements into robust, reusable, and cost-efficient data solutions. Key responsibilities Design, develop, test, and maintain scalable data engineering solutions on Microsoft Azure and Databricks, using Python, PySpark, Scala, and SQL Implement reliable batch and streaming data workflows using Azure Databricks, Delta Lake, and modern data lakehouse patterns Develop data processing applications for ingestion, transformation, enrichment, validation, and serving of analytical datasets Build CI/CD pipelines for data applications using Azure DevOps YAML pipelines and apply software engineering best practices: modular design, clean code, Version Control, code reviews, and documentation Develop reusable libraries, utilities, and frameworks to accelerate data product delivery and support less experienced engineers through coaching and knowledge sharing Collaborate with Cloud, DevOps, and Platform teams on deployment, environment configuration, and operational readiness Technical skills: 5+ years of hands-on experience in data engineering, application development, or cloud-based data platform development Strong hands-on development experience with Python, PySpark, Scala, SQL, and ETL/ELT development Solid experience with the Azure Databricks ecosystem: notebooks, jobs, Spark clusters, Delta Lake, lakehouse architecture, and performance tuning Good knowledge of Azure data services: Azure Data Lake Storage, Azure Data Factory, Azure Key Vault, Azure DevOps, and Azure monitoring/logging capabilities Experience with CI/CD pipelines (Azure DevOps YAML), Version Control, automated testing, modular development, and code reviews Understanding of cloud security, access management, observability, and cost-aware development practices Working knowledge of TerraForm or infrastructure-as-code concepts is a plus Preferred qualifications: Experience with large-scale distributed data processing using Spark and streaming/real-time ingestion patterns Experience building reusable data engineering frameworks, libraries, or shared components Knowledge of data quality frameworks, metadata management, lineage, and governance practices Experience integrating data pipelines with AI, Machine Learning, or analytics use cases; familiarity with MLOps is an advantage Preferred certifications: Microsoft Azure Data Engineer Associate, Azure Developer Associate, Azure Solutions Architect Expert, Databricks Certified Data Engineer Associate/Professional, or Databricks Certified Developer for Apache Spark Soft skills: Strong communication and documentation skills, with the ability to explain technical data engineering concepts to diverse audiences Strong problem-solving mindset with a focus on automation, reliability, and maintainability Team-oriented, proactive, and comfortable working in a cross-functional environment Continuous improvement mindset and willingness to adopt evolving data platform technologies