Freelance Opportunity – Senior Databricks Platform …, Thiruvananthapuram

ThreatXintel Cyber Labs LLP is seeking a highly experienced Senior Databricks Platform Practitioner to join a healthcare data engineering initiative. This is a hands-on, player-coach role where you will actively contribute to delivery while mentoring engineering teams and enforcing platform best practices. ???? About the Role This is not a traditional consulting position. You will work as an embedded member of the engineering pod, participating in daily standups, sprint planning, code reviews, architecture discussions, pair programming, and delivery activities. The ideal candidate is highly hands-on and capable of designing, building, optimizing, and troubleshooting enterprise-scale Databricks Lakehouse solutions while helping engineering teams adopt best practices. ???? Key Responsibilities
• Build and optimize Databricks Lakehouse solutions using PySpark and Delta Lake
• Design metadata-driven ingestion frameworks using control tables and reusable pipeline patterns
• Implement Bronze, Silver, and Gold Medallion Architecture data pipelines
• Develop CDC-based ingestion and transformation frameworks
• Implement inline data quality, governance, and observability controls within pipelines
• Review pull requests, mentor engineers, and provide architectural guidance
• Troubleshoot production issues involving Spark optimization, Auto Loader, merge operations, and distributed systems
• Contribute to CI/CD implementation and deployment automation
• Collaborate closely with agile delivery pods to accelerate platform adoption ???? Required Skills
• 5+ years of hands-on Data Engineering experience
• Robust expertise in Databricks Platform Engineering
• Deep experience with PySpark, Delta Lake, Unity Catalog, and Databricks Workflows
• Strong knowledge of Medallion Architecture (Bronze / Silver / Gold)
• Experience building metadata-driven ingestion frameworks
• Expertise in CDC processing and enterprise data integration patterns
• Experience implementing data quality and governance directly within pipelines
• Strong Spark performance tuning and optimization skills
• Experience with Azure cloud environments
• Familiarity with CI/CD pipelines and Git-based development workflows
• Experience working in Agile pods or squad-based delivery models ???? Preferred Skills
• Healthcare data engineering experience
• Experience with Databricks Lakehouse implementation at enterprise scale
• Experience with Fivetran or similar ingestion tools
• Exposure to DataHub or other modern data catalog/governance platforms
• Experience mentoring engineers and conducting code reviews Apply on Kit Job: kitjob.in/job/4naxkz