Python Developer

PySpark Developer Job DescriptionAbout The Company

Tata Consultancy Services (TCS) is a global leader in IT services, consulting, and business solutions. Established in 1968, TCS has grown to become one of the most respected organizations in the technology sector, with a presence in over 50 countries. The company is committed to helping clients achieve their business objectives by providing innovative, best-in-class consulting, IT solutions, and services. TCS emphasizes a culture of continuous learning, diversity, and inclusion, fostering an environment where employees are empowered to excel and make a meaningful impact. With a focus on sustainability and corporate responsibility, TCS strives to deliver value to its clients, employees, and communities worldwide.

About The Role

We are looking for a talented and experienced PySpark Developer to join our dynamic data engineering team. The ideal candidate will play a crucial role in designing, developing, and maintaining large-scale data processing pipelines that support critical business insights and decision-making processes. This role requires a deep understanding of big data technologies, especially Spark-based frameworks, and the ability to work collaboratively with cross-functional teams including data analysts, data engineers, and business stakeholders. The successful candidate will contribute to building robust, scalable, and high-performance data solutions that align with TCS's commitment to excellence and innovation.

Qualifications

The ideal candidate should possess a strong educational background in Computer Science, Information Technology, or related fields, with a proven track record of hands-on experience in data engineering roles. Relevant certifications in big data or cloud platforms are a plus. Candidates should demonstrate expertise in Python and PySpark, along with solid knowledge of SQL and data warehousing concepts. Experience working with Hadoop ecosystem components such as Hive, HDFS, and YARN is essential. Familiarity with data formats like Parquet, Avro, and JSON, as well as version control tools like Git, is required. Exposure to cloud platforms such as AWS, Azure, or GCP will be advantageous. Excellent problem-solving skills, attention to detail, and the ability to work under tight deadlines are critical for success in this role.

Responsibilities

The PySpark Developer will be responsible for a wide range of tasks to support TCS's data infrastructure. Key responsibilities include designing, developing, and maintaining scalable ETL/ELT pipelines utilizing PySpark, ensuring efficient data processing and transformation. The developer will optimize Spark jobs for maximum performance, reliability, and scalability, while processing large datasets using Spark SQL, DataFrames, and RDDs. They will develop both batch and real-time data processing solutions to meet evolving business needs. Integration of data pipelines with various data platforms such as Hive, HDFS, Snowflake, and Redshift will be a core part of the role. The candidate will collaborate closely with data engineers, analysts, and business stakeholders to understand requirements and deliver solutions accordingly. Monitoring data pipelines for issues, troubleshooting, and ensuring adherence to SLAs is essential. Additionally, the developer will follow best coding practices, participate in version control, and support CI/CD processes. Familiarity with Hadoop ecosystem tools and cloud platforms like AWS, Azure, or GCP will be leveraged to enhance data solutions.

Benefits

Joining TCS offers a comprehensive benefits package designed to support your professional and personal growth. Employees are entitled to competitive salary packages, health insurance, and retirement benefits. TCS promotes work-life balance through flexible work arrangements and ample leave policies. The company invests significantly in employee development through training programs, certifications, and continuous learning opportunities. Employees also benefit from a collaborative and inclusive work environment that encourages innovation and creativity. Additionally, TCS provides opportunities for international assignments and exposure to cutting-edge technologies, fostering a rewarding career path for ambitious professionals.

Equal Opportunity

Tata Consultancy Services is an equal opportunity employer. We are committed to creating an inclusive environment where all employees are valued and respected regardless of race, gender, age, religion, disability, or sexual orientation. We believe diversity drives innovation and excellence, and we actively promote policies that support equal employment opportunities. All qualified candidates are encouraged to apply, and TCS ensures a fair and transparent recruitment process for everyone.