Data Engineer

Mandatory Skills-

• Design and develop data-ingestion frameworks, real-time processing solutions, and data processing and transformation frameworks leveraging open source tools and data processing frameworks.
• Hands-on on technologies such as Apache Nifi, PostgresSQL, DBT, GCP.
• Experience in GCP Cloud Composer, Big Query, DataProc.
• Offer system support as part of a support rotation with other team members.
• Operationalize open-source data-analytic tools for enterprise use.
• Ensure data governance policies are followed by implementing or validating data lineage, quality checks, and data classification.
• Understand and follow the company development lifecycle to develop, deploy and deliver the solutions.

Preferred experience includes-

• Design and develop data-ingestion frameworks, real-time processing solutions, and data processing and transformation frameworks.
• Deploy and provide support for application codes and analytical models.
• Create and enhance data solutions that enable seamless integration and flow of data across the data ecosystem.
• Provide business analysis and develop ETL code and scripting to meet all technical specifications and business requirements according to the established designs.
• Develop real-time data ingestion and stream-analytic solutions leveraging technologies such as Kafka, Apache Spark (SQL, Scala, Java), Python and Hadoop Platform.