Backend Software Developer Jobs in Palo Alto, CA

What You'll Do

• Architect Distributed Systems: Build scalable backend, infrastructure, and agentic services for web, mobile, and multi-platform products.
• Evolve the Agent Runtime: Design and optimize the core execution loop that handles agent reasoning, tool-use frameworks, function calling, memory retrieval, and multi-step orchestration.
• Design Real-Time Architecture: Own and scale real-time messaging infrastructure, event-driven architectures, WebSocket connections, and pub/sub patterns for throughput, latency, and reliability.
• Implement Core AI Capabilities: Optimize LLM integrations, multi-provider model routing (Claude, GPT, Gemini, open-source), context window management, cost optimization, and streaming responses.
• Build Memory & Retrieval Systems: Design semantic search and vector-based embedding infrastructure to handle long-term memory, working memory, and episodic recall for autonomous agents.
• Own Backend Logic End-to-End: Drive database modeling (SQL/NoSQL), API design, performance tuning, and production reliability for high-traffic pipelines.
• Drive Technical Excellence: Write RFCs, evaluate complex technical trade-offs, mentor junior engineers through code reviews, and build alignment across engineering and product teams.

What We're Looking For

• Experience: 5+ years of software engineering experience building production services at scale, with 2+ years hands-on with LLM-based orchestration, multi-agent systems, or agentic solutions.
• Backend Mastery: Deep proficiency in modern backend technologies (Node.js, Python, Go) and frameworks (Express, FastAPI, TypeScript, etc.).
• Systems & Infra Knowledge: Strong understanding of distributed systems, event-driven microservices, message queues, cloud infrastructure (AWS/GCP), Kubernetes, and CI/CD workflows.
• AI & Agentic Expertise: Solid grasp of LLM capabilities and limitations, prompt engineering (system prompts, chain-of-thought, structured output), tool-use execution, and embedding models.
• Real-Time Patterns: Comfort designing and debugging real-time streaming pipelines, long-polling, and highly concurrent networking setups.
• Product & Data Intuition: Deep understanding of database design and the product sense needed to make an AI system feel "alive" and responsive.
• Mindset: Ownership mentality—identify systemic bottlenecks and ship solutions without waiting for exact specifications. Strong communication and a collaborative, team-first attitude.

Nice to Have

• Experience with multi-modal AI architectures (image generation, TTS, speech-to-text, video generation)
• Experience with agent frameworks (LangChain, CrewAI, AutoGPT) or building custom, high-performance execution runtimes
• Experience with fine-tuning, RLHF, or DPO pipelines
• Background in multi-tenant SaaS or internal tooling and operational automation
• Previous startup experience—comfortable with ambiguity and rapid experimentation
• Competitive coding background (IOI, ICPC, Olympiad medalists, etc.)

What We Offer

• Competitive salary in the AI industry
• Equity in a fast-growing startup shaping the future of AI
• Comprehensive health benefits, monthly stipends, company retreats
• A supportive and collaborative office culture—we’re all building and launching together

Pay: $180,000.00 - $300,000.00 per year

Benefits:

• 401(k)
• Dental insurance
• Flexible schedule
• Health insurance
• Paid time off
• Relocation assistance
• Vision insurance

Experience:

• AI Agent Development: 1 year (Required)
• Backend Development: 5 years (Required)

Ability to Commute:

• Palo Alto, CA 94302 (Required)

Work Location: In person