Data Engineer

Job description

This will be data engineer position. Detail JD as follow

Key Responsibilities:

• Data Infrastructure Development: Design, implement, and maintain scalable data pipelines and data warehousing solutions using SQL and Python.
• Data Integration: Collaborate with cross-functional teams to integrate data from various sources (internal systems, APIs, third-party tools) into our data ecosystem.
• ETL Processes: Develop and optimize ETL processes for the extraction, transformation, and loading of large data sets.
• API Integration: Develop and maintain APIs for data ingestion, ensuring seamless data flow between systems and platforms.
• Database Management: Design and manage relational and non-relational databases, ensuring high availability, security, and performance.
• Data Quality Assurance: Implement data validation checks to ensure the integrity and accuracy of data across systems.
• Automation & Optimization: Automate repetitive data tasks and optimize data workflows for better performance and scalability.
• Collaboration: Work closely with other teams to provide the data infrastructure support as needed

Required Skills & Qualifications:

• Strong proficiency in Python with experience in libraries such as Pandas, NumPy, and SQLAlchemy.
• Extensive experience with SQL and relational database management systems (RDBMS), such as PostgreSQL, MySQL, or MS SQL Server.
• Experience with API integration for data ingestion and communication between different systems (RESTful APIs, JSON, OAuth).
• Solid understanding of data modeling and database design concepts.
• Experience with ETL frameworks and data integration techniques.
• Familiarity with cloud platforms such as AWS, GCP, or Azure is a plus.
• Knowledge of data warehousing concepts and technologies like Redshift, Snowflake, or BigQuery is a plus.
• Strong problem-solving and debugging skills with the ability to troubleshoot data issues across different systems.