We are looking for a Back-End Developer to build new capabilities for SuperZoom, CBRE data quality platform.
This role will focus on developing a backend pipeline that processes CBRE client asset files, runs data profiling and data quality checks, and integrates LLM-based assessments via API calls (e.g., ChatGPT). The ideal candidate has experience with data processing, API integrations, and scalable backend architecture.
Key Responsibilities
- Develop a backend pipeline to ingest, profile, and assess client asset data files.
- Implement data profiling and data quality checks using appropriate libraries and frameworks.
- Integrate LLM-based assessments by calling APIs (e.g., OpenAI, Azure AI).
- Optimize processing for large datasets with efficient workflows.
- Collaborate with front-end developers, data quality teams, and business stakeholders.
- Ensure security and compliance in handling client data.
- Maintain scalability and performance in cloud or on-prem infrastructure.
Required Skills Experience
- 5+ years of experience in backend development, focusing on data processing and API integration.
- Strong Python skills, with experience in Pandas, PySpark, or similar libraries for handling large datasets.
- Proficiency in SQL and experience with PostgreSQL, Snowflake, or MS SQL Server.
- Experience integrating with LLM APIs (e.g., OpenAI, Azure AI) for text-based analysis.
- Hands-on experience with RESTful APIs and authentication methods (OAuth, API keys).
- Strong understanding of data quality and profiling techniques.
- Knowledge of Docker and containerized deployments.
Nice-to-Have Skills
- Experience with Apache Airflow for workflow orchestration.
- Exposure to cloud platforms (AWS, Azure, GCP) for scalable backend development.
- Experience with vector databases and embedding-based retrieval (e.g., Pinecone, Weaviate).
- Familiarity with LLM fine-tuning or prompt engineering.
Success Metrics
- Successful implementation of the automated data profiling and LLM-based assessment pipeline.
- Scalability and efficiency in processing large client data files.
- Accurate and meaningful AI-driven assessments delivered via API calls.
- On-time delivery of backend features in coordination with the front-end team.
