Security Engineer Jobs in Palo Alto, CA

About Runware

Runware is the AI inference platform that builds and runs its own infrastructure, end to end. Our Sonic Inference Engine has served more than 20 billion requests for over 1 million developers and 500 million end users, with 400,000+ models behind a single integration, at up to 10x lower cost per token than traditional providers.

We are now building the AI Factory that runs it. The Sonic Inference Pod packs 1 MW of custom-designed compute into a 20-foot container with closed-loop, water-free cooling, sited wherever power exists. Pods are live in the US and Europe, and we are scaling to dozens of pods and tens of thousands of GPUs, all operated as one distributed fabric that reroutes traffic the moment a pod goes offline (announcement).

Every one of those pods holds customer models, customer data and millions of dollars of GPUs, sometimes inside a customer's own perimeter or a sovereign jurisdiction. Keeping all of it secure without slowing anyone down is the job.

The role

As Senior Security Engineer, you will own the security of Runware's AI supercomputer: the bare-metal GPU fleet, the Sonic Inference Pods it lives in, the managed Kubernetes clusters we hand to customers, and the control plane that ties it all together across the US and Europe. You will design the controls, build the first version with your own hands, and set the standard for how security is run across every pod.

This is a hands-on senior individual-contributor role. You will threat-model a new pod before it ships, harden the image that every GPU node boots from, prove to an enterprise customer's security team that their model weights never leave their tenant, and when something goes wrong, lead the response and fix the root cause so it never happens again. You will work directly with our CISO, with the inference engine, platform and hardware teams, and with the site teams deploying pods.

Success means a customer workload on a shared pod is isolated as strongly as on a dedicated one, no human has a standing login to a production node, every change to the fleet is reviewed, signed and auditable, and an enterprise security review takes days instead of months.

What you'll do

• Own the security architecture of the fleet: host hardening for bare-metal GPU nodes, immutable images and a signed build pipeline, boot integrity, and the identity, secrets and access model for humans and services across every pod.
• Secure multi-tenancy on shared infrastructure. Design and verify isolation between customer workloads at the network, Kubernetes, host and GPU layers, and produce the evidence that proves it holds.
• Secure the managed Kubernetes offering: cluster hardening, admission control, RBAC, supply-chain controls for images, and a clean, auditable handover of clusters to customers.
• Build security visibility across the fleet: logging, detection, alerting and audit trails from firmware to API, wired into the fleet dashboards and on-call so that an anomaly on one node in one pod is seen within minutes.
• Lead security reviews and threat models for new pods, sites, servers and platform features, with a focus on real-world risk rather than checklists.
• Run incident response for the infrastructure: investigation, containment, recovery and post-incident learning, as part of the on-call rotation.
• Own vulnerability management for the fleet: firmware, kernel, drivers, GPU stack and container images across thousands of nodes, with a patch cadence you can defend to a customer.
• Answer enterprise customers' security due diligence with engineering evidence, and partner with the CISO on the SOC 2 and ISO 27001 controls that touch the infrastructure.
• Design for sovereign and on-premise deployments: data residency, in-country key management, and physical and remote-access controls for a pod sitting on a customer's site.
• Use AI to accelerate the work: automate evidence collection, log triage, configuration audits and runbooks with LLM-based agents, so that a very small team can secure a very large fleet.

What we're looking for

• 8+ years in security or infrastructure engineering, with several of those securing production infrastructure at a company that runs its own hardware at scale.
• A track record of building security controls, not just auditing them. You have taken a hardening standard, an access model or a detection pipeline from design to running across a fleet, and you know what breaks between the two.
• Deep, current, hands-on Linux security expertise: kernel hardening, namespaces and cgroups, seccomp, eBPF, secure and measured boot, TPM-based attestation, and the tooling to see exactly what a process did.
• Strong Kubernetes security: RBAC, admission control and policy engines, network policy, runtime security, image signing and SBOMs, and the ways a container escapes.
• Fluent in network security on real hardware: segmentation and VLAN or VXLAN isolation, firewalls, out-of-band management networks, and the risks that live in BMC, IPMI and switch management planes.
• Experience with identity, secrets and PKI at scale: short-lived credentials, workload identity, hardware-backed keys, and removing standing access without slowing engineers down.
• Fluent in Python or Go, and comfortable writing the tooling, scanners and automation yourself rather than waiting for someone else to.
• You use AI to accelerate your own output every day, for scripting, log analysis, code review, documentation and incident triage, and you have a clear view of how a small security team armed with these tools should operate differently.
• A clear communicator who can explain the same trade-off to an engineer, a customer's CISO and a founder in one afternoon, and who assesses risk pragmatically rather than by the book.
• Able to travel to sites in the US and Europe as needed.

Nice to have

• Experience securing GPU clusters for AI training or inference: the NVIDIA driver and fabric manager stack, GPU multi-tenancy (MIG, time-slicing, per-tenant isolation), confidential computing on GPUs, and protecting model weights at rest and in flight.
• Firmware and hardware security: UEFI, BMC and NIC firmware lifecycle, Redfish, hardware root of trust and supply-chain attestation for servers arriving at a site.
• Hands-on experience with InfiniBand or RoCE fabrics and the security model of RDMA.
• Delivering SOC 2 or ISO 27001 in a production environment, and compliance tooling and evidence automation.
• Offensive experience: red teaming, penetration testing or vulnerability research against infrastructure.
• Experience with sovereign, air-gapped or regulated deployments, including data residency across jurisdictions.
• Physical security of data centres or edge sites: access control, tamper evidence and remote-hands procedures.

What we offer

• A rare mandate: secure an AI supercomputer from the firmware up, at a company that owns every layer of its stack and whose enterprise customers are trusting it with their models, with the backing to match.
• Meaningful stock options.
• We're a remote-first collective, meeting in person twice a year to plan, brainstorm and celebrate wins. We keep core hours for collaboration and calls; outside those, you set your own schedule.
• Our release cycles are fast and intense, and they're followed by real downtime. We expect you to take it.
• Generous paid time off, including vacation, sick days and public holidays.
• Paid maternity, paternity and caregiver leave.
• Twice-yearly company retreats in inspiring locations.