As a Data Engineer, you will be responsible for designing, building, and maintaining the infrastructure and tools necessary for data collection, storage, processing, and analysis. You will work closely with data scientists and analysts to ensure that data is available, accessible, and in a format that can be easily consumed for business insights.
Responsibilities:
- Design, build, and maintain data pipelines to collect, store, and process data from various sources.
- Create and manage data warehousing and data lake solutions.
- Develop and maintain data processing and data integration tools.
- Collaborate with data scientists and analysts to design and implement data models and algorithms for data analysis.
- Optimize and scale existing data infrastructure to ensure it meets the needs of the business.
- Ensure data quality and integrity across all data sources.
- Develop and implement best practices for data governance, security, and privacy.
- Monitor data pipeline performance / Errors and troubleshoot issues as needed.
- Stay up-to-date with emerging data technologies and best practices.
Requirements:
- Bachelors degree in Computer Science, Information Systems, or a related field.
- Experience with ETL tools like Matillion,SSIS,Informatica
- Experience with SQL and relational databases such as SQL server, MySQL, PostgreSQL, or Oracle.
- Experience in writing complex SQL queries
- Strong programming skills in languages such as Python, Java, or Scala.
- Experience with data modeling, data warehousing, and data integration.
- Strong problem-solving skills and ability to work independently.
- Excellent communication and collaboration skills.
- Familiarity with big data technologies such as Hadoop, Spark, or Kafka.
- Familiarity with data warehouse/Data lake technologies like Snowflake or Databricks
- Familiarity with cloud computing platforms such as AWS, Azure, or GCP.
- Familiarity with Reporting tools
Teamwork/ growth contribution
- Helping the team in taking the Interviews and identifying right candidates
- Adhering to timelines
- Intime status communication and upfront communication of any risks
- Tech, train, share knowledge with peers.
- Good Communication skills
- Proven abilities to take initiative and be innovative
- Analytical mind with a problem-solving aptitude
Preferred Qualifications:
- Masters degree in Computer Science, Information Systems, or a related field.
- Experience with NoSQL databases such as MongoDB or Cassandra.
- Familiarity with data visualization and business intelligence tools such as Tableau or Power BI.
- Knowledge of machine learning and statistical modeling techniques.
