Senior Data Engineer
Senior Data Engineer
NewVision Software & ConsultancyEarly Applicant
- Posted a month ago
- Be among the first 10 applicants
Job Description
Sr. Data Engineer (6-9 yrs)
Responsibilities:
- Data masking: implement PII masking pipeline
- Builds & maintain synthetic data generator
- CI/CD pipeline integration with auto-seed data
- Validates generated data against the ground truth – referential integrity, volume accuracy, etc. data validation characteristics.
Technical skills:
- Python (Faker, factory-boy, custom generators) or Node.js
- Database management: SQL Server, PostgreSQL — insert at scale
- CI/CD pipeline integration (GitHub Actions, Azure DevOps)
- Data masking techniques: tokenization, hashing, format-preserving encryption
- API-based data seeding (REST / GraphQL endpoints)
- Performance: generating & loading millions of records efficiently
More Info
Key Skills
performance generating loading millions of records efficiently
factory-boy
API-based data seeding
GitHub Actions
Faker
format-preserving encryption
custom generators
data masking techniques
About Company
Skills:
snowflake , Apis, Power Bi, Json, Sql, Automated Testing, Apache Airflow, Git, Azure Data Factory, Xml, Azure, Python, AWS, Matillion, CI/CD pipelines, dbt, Webhooks, Fivetran
Skills:
PostgreSQL, Apache Spark, Kafka, Apache Airflow, Pandas, Elasticsearch, Python, Polars, Batch and real-time data pipelines, NoSQL databases, Scalable backend systems, OpenSearch, Data models and storage architectures, Distributed data systems, Data processing performance and optimisation, Data pipelines, Machine learning data pipelines, Vector search retrieval
Skills:
Apache Airflow, Pyspark, Spark, Databricks, Azure, Python, Sql, Fabric, ETL pipelines
Skills:
Apache Airflow, Github, Pyspark, Spark, Azure Databricks, Azure Storage, Python, CodeQL, Cycode
Skills:
snowflake , Data Modelling, Power Bi, Tableau, Redshift, Sql, ELT, Database Design, Azure Data Factory, Postgres, Databricks, Python, Etl, Aws S3