Search by job, company or skills

S

Data Engineer

1-4 Years

This job is no longer accepting applications

new job description bg glownew job description bg glow
  • Posted 2 months ago
  • Over 500 applicants

Job Description

  • Design and architect enterprise-scale data platforms, integrating diverse data sources and tools.
  • Develop real-time and batch data pipelines to support analytics and machine learning.
  • Define and enforce data governance strategies to ensure security, integrity, and compliance.
  • Optimize data pipelines for high performance, scalability, and cost efficiency in cloud environments.
  • Implement solutions for real-time streaming data (Kafka, AWS Kinesis, Apache Flink).
  • Adopt DevOps/DataOps best practices for deployment and monitoring.
  • Required Skills:
  • Strong experience in designing scalable, distributed data systems.
  • Programming skills in Python, Scala, or Java.
  • Expertise in Apache Spark, Hadoop, Flink, Kafka, and cloud platforms (AWS, Azure, GCP).
  • Proficiency in data modeling, governance, and warehousing (Snowflake, Redshift, BigQuery).
  • Familiarity with security/compliance standards such as GDPR and HIPAA.
  • Hands-on experience with CI/CD tools (Terraform, CloudFormation, Airflow, Kubernetes).
  • Experience with data infrastructure optimization using tools like Prometheus and Grafana.
  • Nice to Have:
  • Experience with graph databases, real-time analytics, and IoT solutions.
  • Integration of machine learning pipelines.
  • Contributions to open-source data engineering communities.

About Company

We are an Enterprise AI firm focusing on RLHF and Custom AI solutions. We are dual incorporated in the US and India (our India office is in Gachibowli, Hyderabad). We are led by an IIT-IIM founding team, and have a highly skilled talent pool. Think PhDs, Engineers, Artists.

Job ID: 120653127

Similar Jobs

Hyderabad, India

Skills:

JavaPysparkScalaApache SparkSqlGitGcpEtl ToolsData LakeData WarehousingDatabricksAzureAws S3AWSDelta LakeGCP BigQueryLakehouse architecture

Hyderabad, India

Skills:

snowflake SqlData Warehousing ConceptsELTDimensional ModelingAWSEtlAzureGcpData Securityprivacy best practicesdata governance principlesAirflowworkflow orchestration toolsdata pipelinesdbt

Hyderabad, India

Skills:

UnixHadoopOracle SqlScalaApache SparkKafkaPl SqlAutosysSqlHiveGcpNeo4jShell scriptingSparkMongoDBAzurePythonAWSEtlH2O

Hyderabad, India

Skills:

snowflake DatabricksSqlAWSELTEtlApache AirflowPythonKafkaSparkdbtFlink

Hyderabad, India

Skills:

S3RDSNatural Language ProcessingData ArchitectureEmrRedshiftELTLambdaKinesisData GovernanceEtlmachine learning infrastructuredata quality testingAWS data servicesGlueGenerative Artificial Intelligence