Pyspark Developer

Role : PYspark Developer Exp : 4-8 Years Notice period : Immediate Work Location : Mumbai We are seeking a skilled PySpark Developer with 4-8 years of experience in building scalable data processing and analytics solutions. The ideal candidate will have strong expertise in PySpark, Python, SQL, and Big Data technologies, with hands-on experience in designing, developing, and optimizing ETL/Data Engineering pipelines. Key Responsibilities · Design, develop, and maintain scalable data processing solutions using PySpark. · Build and optimize ETL pipelines for large-scale structured and unstructured datasets. · Develop data transformation and ingestion workflows using Python and Spark. · Write efficient and complex SQL queries for data extraction, validation, and reporting. · Collaborate with business analysts, data architects, and stakeholders to understand data requirements. · Perform data quality checks, validation, and troubleshooting of data issues. · Optimize Spark jobs for performance, scalability, and resource utilization. · Work with cloud and distributed computing environments to process large volumes of data. · Participate in code reviews and ensure adherence to coding standards and best practices. · Support production deployments and resolve application issues. Required Skills · 4-8 years of experience in Data Engineering or Big Data Development. · Strong hands-on experience in PySpark and Python. · Expertise in SQL and relational databases. · Experience with Apache Spark ecosystem. · Strong understanding of ETL concepts and data warehousing principles. · Experience working with large datasets in distributed environments. · Knowledge of version control tools such as Git. · Familiarity with Linux/Unix environments. Preferred Skills · Experience with cloud platforms such as AWS, Azure, or GCP. · Knowledge of Hadoop ecosystem components (Hive, HDFS, Yarn). · Experience with orchestration tools such as Airflow. · Exposure to data lake and lakehouse architectures. · Understanding of CI/CD processes and DevOps practices. Educational Qualification · Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field. Nice to Have · Experience in Banking, Financial Services, Insurance (BFSI), Retail, or Telecom domains. · Knowledge of Delta Lake, Databricks, or Snowflake. · Exposure to real-time data processing using Kafka. Keywords: PySpark, Python, Spark SQL, ETL, Data Engineering, Hadoop, Hive, Databricks, Azure Data Factory, Airflow, Kafka, SQL, Cloud, Big Data.