Custom Software Engineer

Project Role : Custom Software Engineer

Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs.

Must have skills : Data Engineering

Good to have skills : NA

Minimum 3 Year(s) Of Experience Is Required

Educational Qualification : 15 years full time education

Roles and Responsibilities

Design, develop, and maintain scalable data pipelines and ETL processes to collect, process, and store large volumes of structured and unstructured data.

Collaborate with data scientists, analysts, and business teams to understand data requirements and deliver reliable data solutions.

Optimize data architecture for performance, reliability, and scalability.

Implement data quality checks and monitor data integrity throughout the pipeline.

Work with cloud platforms (AWS, GCP, Azure) and big data technologies (Spark, Hadoop, Kafka, etc.).

Manage and optimize databases and data warehouses (e.g., Redshift, BigQuery, Snowflake).

Develop automation tools for data ingestion, transformation, and integration.

Document data flows, architecture, and processes.

Ensure compliance with data governance and security policies.

Professional and Technical skills:

Proven experience as a Data Engineer or in a similar role.

Strong programming skills in Python, Pyspark, Java, or Scala.

Experience with SQL and NoSQL databases.

Familiarity with ETL tools and data pipeline frameworks (e.g., Apache Airflow, Luigi).

Hands-on experience with cloud data services (AWS Glue, Google Dataflow, Azure Data Factory).

Knowledge of big data technologies such as Hadoop, Spark, Kafka.

Knowledge of machine learning pipelines and data science workflows.

Familiarity with data visualization tools (Tableau, Power BI).

Experience with containerization and orchestration tools (Docker, Kubernetes).

Additional Information:

Bachelor s degree in Computer Science, Engineering, Information Technology, or related field.

Excellent communication and collaboration skills.

Strong problem-solving skills and attention to detail.

, 15 years full time education