Senior Data Engineer

Description: Build and maintain data pipelines and datasets for reporting, visualization, machine learning, and data-driven decision making across various business functions. Develop scalable data pipelines and reusable datasets within a Databricks lakehouse environment. Ingest, validate, clean, and transform structured and unstructured data using Python, PySpark, and SQL. Build data models and reporting datasets for Tableau and analytics tools like Spotfire. Integrate data warehouse and reporting systems with cloud analytics platforms. Improve pipeline reliability, data quality, and processing performance. Implement automated testing, monitoring, documentation, and CI/CD practices. Collaborate with analysts, architects, developers, and business stakeholders to deliver data solutions. Provide technical guidance, review code, and mentor other data engineers. Requirements: 7+ years of experience in data engineering, including production data pipelines. Strong SQL skills and understanding of data modeling, database design, and data warehousing. Hands-on experience with Databricks, Apache Spark, and cloud data platforms. Proficiency in Python and PySpark. Experience developing ETL/ELT pipelines and integrating data from multiple sources. Experience preparing datasets for Tableau or similar BI tools. Familiarity with source control, automated testing, and CI/CD practices. Ability to troubleshoot complex data issues and optimize pipeline performance. Strong communication skills and experience collaborating with technical teams and stakeholders. Experience with Spotfire, Oracle, Informatica, R, or data virtualization preferred. Databricks certification preferred. Benefits: Opportunity to work with cutting-edge technologies in a collaborative environment. Potential for professional growth and development through mentorship and training.