6-10 years
3 months ago 29 Applied
Job Description

Responsibilities for Lead Data Engineer

•Create and maintain optimal data pipeline architecture,

•Assemble large, complex data sets that meet functional / non-functional

business requirements.

•Build the infrastructure required for optimal extraction, transformation, and

loading of data from a wide variety of data sources using SQL and AWS

‘big data’ technologies.

• Work with stakeholders including the Executive, Product, Data and Design

teams to assist with data-related technical issues and support their data

infrastructure needs.

• Keep our data separated and secure across national boundaries through

multiple data centers and AWS regions.

•Create data tools for analytics and data scientist team members that assist

them in building and optimizing our product into an innovative industry

leader.

• Work with data and analytics experts to strive for greater functionality in our

data systems.

Qualifications for Data Engineer

•Advanced working SQL knowledge and experience working with relational

databases, query authoring (SQL) as well as working familiarity with a

variety of databases.

•Experience building and optimizing ‘big data’ data pipelines, architectures and

data sets.

•Experience performing root cause analysis on internal and external data and

processes to answer specific business questions and identify opportunities

for improvement.


•Build processes supporting data transformation, data structures, metadata,

dependency and workload management.

•A successful history of manipulating, processing and extracting value from

large disconnected datasets.

• We are looking for a candidate with 5-6 years of experience in a Data Engineer

role, who has attained a Graduate degree in Computer Science, Information

Systems or another equivalent branch. They should also have experience

using the following software/tools:

Must haves

•Experience with big data tools: Spark (PySpark), SparkSQL –

Architecture, development and design knowledge.

•Experience with relational SQL and NoSQL databases, including

Oracle, MySQL, Cassandra etc.

•Experience with data pipeline and workflow management tools:

Azkaban, Luigi, Airflow, etc.

•Experience with AWS cloud services: EC2, EMR, RDS, Glue, Athena

•Experience with at least one object-oriented/object function scripting

languages: Python, Java, Scala, etc.

Good to Have

• AWS Associate Level Certification and above preferred.

• AWS Redshift

• Experience in building a Data Lake – understanding of architecture

and building blocks to build a data lake.

The company LEVARAGE BUSINESS SOLUTIONS (LBS) was established in 2018 to serve businesses' staffing requirements. Based on the values of honesty and openness, LBS works to provide its customers with value-added services. At LBS, we firmly think that a company's human resources are its most important and valuable asset. Therefore, our goal is to find the best resources and aid in their integration into the values and ethos of the firm.

Since our inception, we have built a solid reputation for unwavering excellence, dedication, dependability, and efficiency. This is due to the fact that we collaborate closely with businesses to determine the best method for meeting customer demands. We can provide a distinctive approach to a range of issues because of our roots in management consulting provide HR services. The full range of human resource management options is included in our service offering. By meeting their demands, LBS is devoted to 'Redefining Value' for its clients.

About Recruiter
Chinnappa P N

Functions

Customer Service/Call Centre/BPO,
IT

Industry

Other,
Automotive/Automobile/Ancillaries,
Banking/Accounting/Financial Services,
Chemicals/Petrochemicals

Skills/Roles

Voice,
Non Voice,
IT,
Ites,
Banking

LEVEL HIRING FOR

Junior Level,
Mid Level,
High Level

People Also Considered

Career Advice to Find Better