We are seeking a skilled Data Engineer with strong hands-on experience in ClickHouse, Kubernetes, SQL, and Python. You will be responsible for building and maintaining efficient data pipelines, optimizing query performance, and developing APIs to support scalable data services. This role requires expertise in data modeling, container orchestration, and collaboration with data scientists and analysts to ensure data quality and meet business needs.
Roles & Responsibilities:
- Design, build, and maintain scalable and efficient data pipelines and ETL processes.
- Develop and optimize ClickHouse databases for high-performance analytics, including data modeling, query optimization, and performance tuning.
- Create RESTful APIs using FastAPI to expose data services.
- Work with Kubernetes for container orchestration and deployment of data services.
- Write complex SQL queries to extract, transform, and analyze data from PostgreSQL and ClickHouse.
- Collaborate with data scientists, analysts, and backend teams to support data needs and ensure data quality.
- Monitor, troubleshoot, and improve the performance of data infrastructure.
Skills Required:
- Strong experience in ClickHouse (data modeling, query optimization, performance tuning).
- Expertise in SQL, including complex joins, window functions, and optimization.
- Proficient in Python, especially for data processing (Pandas, NumPy) and scripting.
- Experience with FastAPI for creating lightweight APIs and microservices.
- Hands-on experience with PostgreSQL (schema design, indexing, and performance).
- Solid knowledge of Kubernetes (managing containers, deployments, and scaling).
- Understanding of software engineering best practices (CI/CD, version control, testing).
- Experience with cloud platforms like AWS, GCP, or Azure is a plus.
- Knowledge of data warehousing and distributed data systems is a plus.
- Familiarity with Docker, Helm, and monitoring tools like Prometheus/Grafana is a plus.
QUALIFICATION:
- Bachelor's degree in Computer Science, Information Technology, or a related field, or equivalent practical experience.
