Data Engineer
Search Jobs
Search by job, company or skills
Search by job, company or skills
Qualification: BE/B.Tech/M.Tech - Computer/IT
Experience: 3 to 5 Years
Roles & Responsibilities:
•Create and manage data pipeline architecture for data ingestion, pipeline setup and data curation
•Experience working with and creating cloud data solutions
•Assemble large, complex data sets that meet functional/non-functional business requirements
•Implement the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Pyspark, SQL and AWS big data-technologies
•Build analytics tools that use the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics
•Manipulate data at scale: getting data in a ready-to-use state in close alignment with various business and Spoke stakeholders.
•ETL
•Data Lake, Data Warehouse, RDS architectures knowledge
•Python, SQL (Any other OOP language is also valuable)
•Pyspark (preferably) or Spark Knowledge
•Object-oriented programming, Clean Code and good documentation skills
•AWS: S3, Athena, Lambda, Glue, IAM, SQS, EC2, Quicksight, and etc.
•Git
•Data Analysis & Visualization
Additional Skills:
•AWS CDK Cloud Development Kit
•CI/CD knowledge
Job ID: 150672203
Skills:
snowflake , S3, Redshift, Jenkins, Lambda, Spark Streaming, Apache Kafka, Gitlab, Data Governance, Python, AWS, Alation, Glue, Data Quality Mesh
Skills:
.NET, data warehouses , Spark SQL, Power Bi, Azure Log Analytics, Powershell Scripting, Hive, Azure Data Factory, Data lakes, Microsoft Azure Data platform, Azure SQL Data Warehouse, Azure Storage Services, Azure Application Insights, Data Bricks, Stream Analytics, data marts, Event Hubs, Azure SQL DB, Azure Analysis Services
Skills:
Python Frameworks, database systems and data storage solutions, data quality assurance and validation methods, data processing and transformation techniques, data pipeline development and management
Skills:
S3, RDS, Pyspark, Databricks, Emr, Redshift, Python, Sql, AWS, Etl
Skills:
Git, Power Bi, SQL Server, Dax, Dashboard creation, Microsoft Azure Data Factory
