Agentic AI - Senior Engineer I

Job Description

At American Express, our culture is built on a 175-year history of innovation, shared and Leadership Behaviors, and an unwavering commitment to back our customers, communities, and colleagues. As part of Team Amex, you'll experience this powerful backing with comprehensive support for your holistic well-being and many opportunities to learn new skills, develop as a leader, and grow your career.

Here, your voice and ideas matter, your work makes an impact, and together, you will help us define the future of American Express.

We're building next-generation Agentic AI systems that can reason, plan, and act autonomously in complex, real-world environments. This role is for engineers who want to work end-to-end on agentic systems - from how an agent thinks, to how it runs reliably at scale, to how humans trust and control it.

1. Agent Architecture & LLM Engineering (the algo, core)

You will design how agents think, plan, and act. You'll implement, debug, optimize, tune, and maintain agent systems in production.

Must-have skills

• Hands-on experience with LLM orchestration & agent frameworks (LangGraph, OpenAI SDKs, CrewAI, AutoGen, etc.)
• Prompt engineering beyond prompts, including:
  • Tool calling
  • Planning and reflection loops
  • Short-term vs long-term memory design
• RAG & retrieval system design (vector databases, hybrid search, grounding strategies)
• Model evaluation focused on agent behavior, not just static accuracy

You are likely a strong fit if you

• Have built agents that run autonomously for minutes or hours
• Can clearly explain tradeoffs between single-agent and multi-agent systems
• Know when not to use an agent and can justify simpler approaches
• Understand agent architectures deeply and are comfortable debugging and optimizing them in real systems

2. Distributed Systems & Platform Engineering

You'll help build the backend and platform layers that make agentic systems reliable - and fail in boring, predictable ways rather than surprising ones.

Scope

• Infrastructure platforms (GPU & accelerator orchestration, Kubernetes clusters, secure execution sandboxes)
• AI runtime & serving platforms (model serving infrastructure, tool execution environments)
• Developer platforms (SDKs, APIs, CI/CD pipelines)
• Enterprise control planes (security, observability, governance hooks)

Must-have skills

• Designing and operating stateful workflows (event-driven architectures, DAGs, async execution)
• Building observability for agent systems: tracing decisions, tool calls, and failures
• Implementing cost controls and rate limiting
• Designing for horizontal scaling and fault tolerance

Tech stack experience

• Kubernetes and/or serverless platforms
• Queues & orchestration tools (Temporal, Airflow, Step Functions)
• API and service design

You are likely a strong fit if you

• Have operated production systems where things go wrong frequently
• Naturally think about idempotency, retries, backpressure, and failure handling

3. Data & Knowledge Engineering

You'll design the data and knowledge foundations that agents reason over, ensuring accuracy, freshness, and trustworthiness.

Must-have skills

• Designing knowledge graphs or structured semantic layers
• Building data ingestion pipelines for documents, events, APIs, and streaming data
• Defining embeddings strategies and managing their lifecycle
• Ensuring data quality, freshness, and provenance

Bonus

• Experience integrating agents with enterprise systems (CRM, ERP, risk, payments)

4. Safety, Controls & Governance

You'll design agentic systems with safety and control as first-class concerns, especially in regulated or customer-facing environments.

Must-have skills

• Designing and implementing guardrails and policy enforcement
• Building human-in-the-loop workflows
• Defining permissioning and action scopes (what agents are allowed - and not allowed - to do)
• Red-teaming agent behavior and identifying failure modes

You are likely a strong fit if you

• Think in terms of blast radius and containment
• Can clearly articulate how systems fail
• Design systems assuming agents will hallucinate or misbehave

5. Product & UX for AI Systems

You'll help ensure that agentic systems are understandable, trustworthy, and useful to real users.

Must-have skills

• Designing explainability (helping users understand an agent acted)
• Confidence calibration (deciding when agents should act vs ask for help)
• Designing feedback loops from user model system
• Defining metrics beyond usage, such as trust, intervention rate, and task completion

You are likely a strong fit if you

• Have shipped AI-powered features used by non-technical users
• Think in terms of progressive autonomy rather than all-or-nothing automation

6. Evaluation, Experimentation & MLOps

You'll build the measurement and experimentation systems that allow agent behavior to continuously improve.

Must-have skills

• Designing agent-level evaluation frameworks (task success, trajectory quality)
• Building simulation and offline testing environments
• Running A/B tests for agent behaviors
• Operating continuous improvement loops

You are likely a strong fit if you

• Are comfortable saying, this agent is worse - and here's how we know

If you want, I can:

• Make this more senior-level or staff/principal-oriented
• Tighten it into a shorter external posting
• Adapt the language for a highly regulated enterprise audience
• Add a Day in the life or What success looks like in 6 months section

Just tell me the audience and seniority level.

We back you with benefits that support your holistic well-being so you can be and deliver your best. This means caring for you and your loved ones physical, financial, and mental health, as well as providing the flexibility you need to thrive personally and professionally:

• Competitive base salaries

• Bonus incentives

• Support for financial-well-being and retirement

• Comprehensive medical, dental, vision, life insurance, and disability benefits (depending on location)

• Flexible working model with hybrid, onsite or virtual arrangements depending on role and business need

• Generous paid parental leave policies (depending on your location)

• Free access to global on-site wellness centers staffed with nurses and doctors (depending on location)

• Free and confidential counseling support through our Healthy Minds program

• Career development and training opportunities

American Express is an equal opportunity employer and makes employment decisions without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran status, disability status, age, or any other status protected by law.

Offer of employment with American Express is conditioned upon the successful completion of a background verification check, subject to applicable laws and regulations.