About TO THE NEW:
TO THE NEW is a digital technology services company providing innovative product engineering solutions to diverse industries across the globe. We design and build digital platforms with Cloud, Data, and AI as the main pillars.
Job Summary/ Your Role in a Nutshell:
The ideal candidate would be an experienced Big Data Architect with expertise in analytics, data engineering, architecture, governance, and framework design. They should approach business problems with creative thinking and guide teams to success by spearheading design and implementation processes while fostering long-term client relationships. Team handling experience is a must.
What you'll do:
- Act as the Solution Architect for Data Engineering, leading end-to-end architecture design and owning technical decision-making across Big data platforms such as Databricks, Snowflake, and modern data ecosystems.
- Drive architecture discussions with clients and stakeholders, evaluating trade-offs, defining data architecture, and establishing governance, scalability, and performance standards.
- Design and oversee end-to-end data solutions, ensuring alignment with business objectives, enterprise architecture, and best practice.
- Own project planning and execution, including effort estimation, resource planning, timelines, milestones, risks, and assumptions.
- Lead and manage delivery teams to ensure successful, high-quality implementation of data platforms within defined timelines and client expectations.
- Ensure successful delivery of high-quality solutions, enforcing best practices in performance, governance, and reliability while driving continuous improvement
- Communicate a compelling and inspired vision to all customer levels, from CTOs/CIOs to engineering managers and programmers
- Leverage extensive knowledge of distributed systems, process flows, and procedures to aid analysis and recommendations for solution offerings
- Maintain a strong network and promote the organization at various meetings, forums, panels, publications, and conferences to establish thought leadership in the industry
- Responsible for POCs on any tech stacks introduced, or possible version upgrades
What you need:
- BE/B.Tech/MCA with at least 12+ years of experience in design and development using various ETL, database technologies with at least 5+ years associated Data Engineering technology stack and programming languages
- Hands-on experience in 4 or more areas:
- Databricks
- Spark Streaming, Spark SQL, Spark ML
- Kafka/Flume
- AWS/ Azure / GCP
- Apache NiFi
- Hadoop, HDFS, MR
- Data Governance
- Visualization and Reporting frameworks like D3.js, Zeppelin, Grafana, Kibana Tableau, Pentaho, PowerBI
- NoSQL Databases HBase/Cassandra/Neo4j/MongoDB
- ElasticSearch
- Data security (Kerberos/OpenLDAP/Knox/Ranger)
- Extensive knowledge of the current landscape and the ability to visualize technology and industry trends
- Working knowledge of Big Data Integration with Third-party/in-house built Metadata Management, Data Quality, Master Data Management solutions, structured/unstructured data
- Ability to successfully lead a team of big data engineers and big data developers through all phases of the development life cycle, including requirements definition, architecture, design, development, implementation, and testing
- Active involvement in the community in terms of articles,blogs or speaking engagements at conferences
