Teamware Solutions is seeking a talented and passionate Big Data Engineer with 2-4 years of experience to join our dynamic team. This role is crucial for designing, developing, implementing, and troubleshooting scalable Big Data solutions that process, store, and manage large volumes of diverse data. The successful candidate will contribute significantly to ensuring smooth data operations and enabling data-driven insights to meet various business objectives for our clients.
Roles and Responsibilities:
- Data Pipeline Development: Design, build, and maintain robust and scalable ETL (Extract, Transform, Load) pipelines for ingesting, transforming, and loading large datasets from various sources into Big Data platforms.
- Big Data Ecosystem Utilization: Work with and optimize components of the Hadoop ecosystem (e.g., HDFS, Hive) and Apache Spark for distributed data processing and analysis.
- Data Processing & Transformation: Develop efficient code using programming languages (e.g., Python, Scala, Java) to perform data manipulation, cleansing, and transformation for analytical purposes.
- Database Management: Interact with and manage various databases, including relational (SQL) and NoSQL databases (e.g., Cassandra, MongoDB, HBase), for data storage and retrieval.
- Troubleshooting & Optimization: Identify, troubleshoot, and resolve performance bottlenecks and issues within Big Data pipelines and infrastructure.
- Data Quality & Governance: Assist in implementing data quality checks and contribute to data governance practices to ensure data accuracy, consistency, and reliability.
- Collaboration: Collaborate effectively with Data Scientists, Data Analysts, and other engineering teams to understand data requirements and deliver reliable data solutions.
- Monitoring & Support: Monitor Big Data jobs and processes, providing operational support and maintenance for developed solutions.
- Continuous Learning: Stay updated with the latest trends and advancements in Big Data technologies and contribute to the adoption of new tools and best practices.
Preferred Candidate Profile:
- Experience: 2 to 4 years of hands-on experience in Big Data engineering or a related data-centric role.
- Big Data Technologies: Practical experience with key Big Data technologies such as Apache Hadoop, Apache Spark, and data warehousing concepts (e.g., Hive).
- Programming Skills: Proficiency in at least one programming language relevant to Big Data (e.g., Python, Scala, or Java).
- Database Skills: Strong command of SQL for data querying and manipulation. Familiarity with NoSQL databases is a plus.
- Cloud Exposure (Plus): Exposure to Big Data services on cloud platforms (e.g., AWS EMR, Azure Data Lake Analytics, Google Cloud Dataproc) is advantageous.
- Problem-Solving: Excellent analytical and problem-solving skills with a keen eye for detail in complex data environments.
- Communication: Good verbal and written communication skills to effectively collaborate with team members and stakeholders.
- Education: Bachelor's degree in Computer Science, Engineering, Data Science, or a related quantitative field.
