Key Responsibilities:
Data Engineering & Pipelines
- Develop and maintain robust data pipelines for large and complex datasets.
- Extract, transform, and load (ETL) data from various sources and file formats.
- Handle time series data and optimize pipelines for performance.
- Monitor, troubleshoot, and ensure optimal operation of data systems.
- Perform root cause analysis for any production issues.
Technology & Tools
- Hands-on coding in Python; familiarity with .NET or Kafka is a plus.
- Experience with Docker and Kubernetes for containerization and orchestration.
- CI/CD pipelines and Git for version control.
- Data modeling, workflow management, and large-scale dataset handling.
- Optional: Dashboard development using Grafana or similar reporting tools.
Collaboration & Communication
- Work closely with development teams and product owners to deliver high-quality solutions.
- Communicate effectively across teams and contribute to collaborative problem-solving.
- Participate in agile development processes and code reviews.
