Skip to main content
T

Data Engineering Tech Lead

Tcs Ct Usa
13 hours ago
Contract
Remote
$65 - $70 USD hourly
Automation

Roles & Responsibilities

  • Understand business requirements, data domains, and product objectives to develop scalable data engineering solutions.

  • Design and implement data transformation pipelines to derive business metrics, KPIs, and analytical datasets for Global Product teams.

  • Develop and maintain scalable Databricks-based data engineering solutions using Spark and Scala.

  • Build, optimize, and support Data Lake/Lakehouse architectures on AWS.

  • Implement data ingestion, transformation, aggregation, and publishing processes using Databricks and Spark.

  • Ensure data accuracy, completeness, timeliness, and availability of enterprise datasets.

  • Monitor, troubleshoot, govern, and support production data pipelines and data platforms.

  • Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency.

  • Implement data quality controls, validation frameworks, and automated monitoring solutions.

  • Collaborate with Data Architects, Product Owners, Analysts, and Business Teams to deliver data products.

  • Deploy and manage data solutions using standardized DevOps and CI/CD pipelines.

  • Participate in code reviews and implement engineering best practices for reliability, maintainability, and security.

  • Create and maintain technical documentation, operational procedures, and support guidelines.

  • Drive continuous improvement, design, and standardization of processes and methodologies.

  • Assess feasibility, complexity, and scope of new capabilities and solutions.

  • Lead cross-functional stakeholder collaboration and communication.

  • Support Customer Data Strategy and Roadmap Development.

    Technical Skills

    • Strong hands-on experience with Databricks for data engineering, data transformation, and pipeline orchestration.

    • Expertise in Apache Spark (PySpark/Spark SQL) and Scala for large-scale distributed data processing.

    • Strong experience with AWS Cloud Platform, including data and analytics services.

    • Experience designing and implementing enterprise Data Lake/Lakehouse solutions.

    • Expertise in building and maintaining ETL/ELT pipelines using Databricks and Spark.

    • Strong SQL skills with large structured and semi-structured datasets.

    • Experience with Delta Lake, data governance, and data quality frameworks.

    • Hands-on experience with CI/CD pipelines, GitHub Actions, Jenkins, or similar DevOps tools.

    • Experience with data monitoring, troubleshooting, performance tuning, and production support.

    Functional Skills

    • Business requirements analysis and data domain understanding.

    • Data modeling and business metric/KPI development.

    • Data governance, data quality, and regulatory compliance.

    • Root cause analysis and issue resolution.

    • Understanding of Agile methodologies and collaboration with global business and product teams.

      Experience

      • 7+ years of overall experience

      • 5–8+ years of Data Engineering experience

      • 3+ years of Databricks experience in enterprise-scale environments

      Education

      • Bachelor’s in Engineering