Senior Python / PySpark Data Engineer
Job Description
We are seeking a Senior Python / PySpark Data Engineer with strong software engineering fundamentals and hands-on experience building scalable data solutions.
Required Skills
- Strong Python development experience with OOP, reusable code, classes, and functions
- Experience building and deploying Python packages (PyPI, wheel files)
- Expertise in Unit Testing and Integration Testing
- Strong CI/CD experience using GitHub Actions
- Solid PySpark knowledge, including:
- Performance tuning and optimization
- Partitioning and data ordering
- JDBC tuning and large-scale data ingestion
- Understanding of security best practices, access controls, and secure data access
Nice to Have
- Experience with enterprise data platforms
- Cloud and modern data engineering experience
What You'll Do
- Develop scalable Python and PySpark applications
- Build reusable frameworks and libraries
- Implement automated testing and CI/CD pipelines
- Optimize large-scale data processing workloads
- Collaborate with engineering teams to deliver secure, high-quality solutions
More Info
Key Skills
Large-scale data ingestion
Secure data access
Access controls
CI CD
GitHub Actions
Security best practices
JDBC tuning
Data ordering
About Company