Sr. Data Engineer
Sr. Data Engineer
Insight GlobalThis job is no longer accepting applications
Job Description
Key Responsibilities
- Design and develop real-time data pipelines using Kafka, Spark, AWS Kinesis, and related technologies.
- Build and support scalable data processing solutions on AWS.
- Modernize legacy ingestion frameworks using cloud-native services.
- Develop and optimize large-scale distributed data processing systems.
- Troubleshoot and support production data platforms and streaming applications.
- Collaborate with cross-functional teams on enterprise data integration initiatives.
Required Skills & Experience
- 10+ years of experience in Data Engineering, Software Engineering, or related fields.
- Strong expertise in AWS Cloud Services and cloud-native architectures.
- Hands-on experience with Apache Spark (Scala & PySpark).
- Strong knowledge of the Hadoop Ecosystem and distributed computing.
- Experience with Kafka, streaming technologies, and real-time data pipelines.
- Experience with AWS Kinesis, Kinesis Firehose, Flume, or similar ingestion platforms.
- Strong programming skills in Python, Scala, and Java.
- Experience with Big Data, Enterprise Data Warehousing, and Distributed Systems.
More Info
Key Skills
AWS Kinesis
Kinesis Firehose
AWS Cloud Services
Enterprise Data Warehousing
About Company
Skills:
snowflake , Apache Airflow, Power Bi, Python, Sql, dbt
Skills:
Maven, Pyspark, Apache Spark, Sql, Apache Airflow, Jenkins, Git, Gcp, Azure, Python, AWS, Etl, Subversion
Skills:
Sql, Pyspark, Unix Shell Scripting, Maven, Spring Boot, ELT, Jenkins, Gradle, Docker, Python, AWS, Etl, Java 8-17
Skills:
data engineering , Aws Cloud, Tableau, Data Analytics, Apache, Sql, Python, Airflow
Skills:
Python, Machine Learning, Deep Learning, Natural Language Processing, Tensorflow