Greeting from NCS!
NCS group is in an exciting phase of growth and transformation journey. NCS is a leading technology services firm with a presence in Asia Pacific and partners with governments and enterprises to advance communities through technology. Combining the experience and expertise of its 10,000-strong team across 55 specializations, NCS provides differentiated and end-to-end technology services to clients with its NEXT capabilities of digital, cloud, platforms as well as core offerings in application, infrastructure, engineering, and cyber security.
Create the extraordinary and impact millions every day. Be part of our multi-talented team and create meaningful work across different industries as we innovate for our tomorrow. If you have an entrepreneurial spirit with big ideas and are excited about creating technological innovations that impact millions of lives, come and be part of the company that makes the ordinary, extraordinary, and if you are passionate about new technologies and new ideas, NCS could be a place for you!
Sr. Data Engineer
RESPONSIBILITIES
- Designing and implementing data ingestion pipelines from multiple sources using SQL,Azure Data bricks.
- Developing scalable and re-usable frameworks for ingesting data sets
- Integrating the end-to-end data pipeline - to take data from source systems to target data repositories ensuring the quality and consistency of data is always maintained.
- Working with event-based/streaming technologies to ingest and process data
- Working with other members of the project team to support the delivery of additional project components (API interfaces, Search)
- Evaluating the performance and applicability of multiple tools against customer requirements
REQUIREMENTS
- Minimum 5 years of experience in data analytics field
- Experience with Azure/AWS Databricks Experience in building and optimizing data pipelines, architectures and data sets
- Excellent experience in Scala or Python, PySpark and SQL Ability to troubleshoot and optimize complex queries on the Spark platform
- Knowledgeable on structured and unstructured data design / modelling, data access and data storage techniques
- Expertise in designing and deploying data applications on cloud solutions, such as Azure or AWS
- Hands on experience in performance tuning and optimizing code running in Databricks environment
- Demonstrated analytical and problem-solving skills particularly those that apply to a big data environment
