Custom Software Engineer

Project Role : Custom Software Engineer

Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs.

Must have skills : PySpark

Good to have skills : NA

Minimum 3 Year(s) Of Experience Is Required

Educational Qualification : 15 years full time education

Summary:

As a Custom Software Engineer, a typical day involves creating tailored software solutions by designing, coding, and improving various components within systems or applications. The role requires working with modern development frameworks and following agile methodologies to ensure the delivery of scalable and efficient software that meets unique business requirements. Collaboration with team members and stakeholders is an integral part of the daily routine to align development efforts with project goals and timelines.

Roles & Responsibilities:

• Strong experience in designing, developing, and maintaining ETL pipelines for large-scale data processing.
• Hands-on experience with data extraction, transformation, and loading from various structured/unstructured sources.
• Experience with data quality checks, data validation, and data lineage tracking.
• Knowledge of incremental loads, CDC (Change Data Capture), and data partitioning strategies.
• Data Processing: pandas, numpy,
• Data Extraction & Loading: boto3 (AWS SDK), requests,
• Database Connectivity: psycopg2, pymysql
• Testing: pytest, unittest
• The candidate should have a minimum of 5-6 years of experience in Pyspark
• AWS certifications at the Professional or Specialty level will be an added advantage
• Expected to perform independently and become an SME.
• Required active participation/contribution in team discussions.
• Contribute in providing solutions to work related problems.
• Collaborate with cross-functional teams to understand business needs and translate them into technical solutions.
• Maintain and enhance existing software components to improve performance and reliability.
• Document development processes and solutions to support knowledge sharing and future maintenance.
• Assist junior team members by providing guidance and support to foster their professional growth.
  
  

Professional & Technical Skills:

• Experience in PySpark for distributed data processing on Hadoop or Spark clusters.
• Ability to write optimized Spark SQL, DataFrame, and RDD transformations.
• Strong experise using PySpark ETL libraries -pyspark.sql, pyspark.sql.functions
• Strong expertise in Python for ETL development.
• Must To Have Skills: Proficiency in PySpark.
• Experience in developing scalable software solutions using distributed computing frameworks.
• Strong problem-solving skills with the ability to analyze and optimize code performance.
• Familiarity with agile software development practices and continuous integration/continuous deployment pipelines.
• Ability to work effectively in a collaborative team environment and communicate technical concepts clearly.
• Knowledge of data processing and transformation techniques within big data ecosystems.
  
  

Additional Information:

• The candidate should have minimum 3 years of experience in PySpark.
• This position is based at our Bengaluru office.
• A 15 years full time education is required.
  
  

, 15 years full time education