N
Data Engineer
N
Data Engineer
NETAPPThis job is no longer accepting applications
Job Description
Job Summary
We are seeking an IC3 Data Engineer (5-8 years) to design, build, and operate the data platforms that power our GenAI applications. This is a hands-on, pureplay data engineering role with strong database and systems administration ownership: you will build and tune batch and streaming pipelines, administer databases and jobs, and keep the platform reliable and performant. You will supply clean, governed data - including embeddings and vector data - to retrieval, chatbot, and agent workloads. Our stack is cloud-agnostic experience with any major cloud is welcome.
Job Requirements
- 5-8 years in data engineering, platform engineering, or analytics engineering.
- Strong SQL and hands-on database/query performance optimization (OLTP and OLAP).
- Strong Python, Spark/PySpark, ETL/ELT, Kafka streaming, and multi-source data integration.
- Database administration: access control, backups, replication, high availability, and job management.
- Systems administration: Linux, OS patching, certificates, monitoring, backup, and disaster recovery.
- Hands-on MongoDB familiarity with Cassandra and vector databases (Milvus, Qdrant, or pgvector).
- Workflow orchestration (Apache Airflow or similar) data modeling and warehousing.
- Power BI (Power Query, DAX) and Git.
Education
- Bachelor's in Computer Science or a related field (or equivalent experience).
- 5-8 years of professional experience in data engineering and data platform operations.
More Info
Job Type:
Industry:
Function:
Employment Type:
Key Skills
About Company
Skills:
Aws Redshift, Data Modeling, Automated Testing, Scala, Apache Spark, ELT, Azure Synapse, Gcp, Databricks, Data Warehousing, Azure, Python, AWS, Etl, Lakehouse Architecture, Google BigQuery, Unity Catalog, Delta Lake, CI/CD, Infrastructure-as-Code, Observability, Databricks Workflows
Skills:
RAG (Retrieval-Augmented Generation), Azure Data Factory (ADF), Pyspark, Azure Sql, Sql, ELT, Devops, Azure Synapse Analytics, Azure Data Lake, Spark, Cosmos DB, Python, Etl, Fabric Lakehouse architectures, Infrastructure as Code, vector databases, CI/CD pipelines, Microsoft Fabric, Azure AI Search
Skills:
snowflake , Java, BigQuery, Scala, Kafka, Redshift, Sql, Git, Gcp, Kinesis, Spark, Azure, Python, AWS, Airflow, Alation, Flink, Atlan, Prefect, dbt, Dagster, CI/CD
Skills:
Large Language Models (LLM), Pyspark, Azure Databricks, Sql, Rest Apis, Visual Studio Code, Python, Azure DevOps, Generative AI, Cloud-based data platforms, Scalable data architectures, ETL processes, Monitoring and alerting systems
Skills:
Pyspark, Azure Databricks, Data Modeling, Sql, ELT, Databricks, Etl, GenAI frameworks and platforms, Databricks Mosaic AI, Lakehouse Architecture, Medallion Architecture, Transformation Notebooks, Batch and real-time streaming data processing frameworks, Cloud-based data storage solutions