Infrastructure Software Engineer Jobs in New York, NY
Infrastructure Software Engineer jobs in New York, New York are concentrated in Midtown Manhattan, the Financial District, and Hudson Yards, with strong demand across financial services, media technology, and enterprise cloud infrastructure. Employers hiring right now include Google, Meta, and Bloomberg. Find a role that fits below and apply directly.
Find JobsOverview
Showing 4 of 44+ Infrastructure Software Engineer jobs
Note: By applying to this position you will have an opportunity to share your preferred working location from the following: Sunnyvale, CA, USA; New York, NY, USA.
Minimum qualifications:
- Bachelor's degree or equivalent practical experience.
- 8 years of experience programming in C++.
- 5 years of experience with design and architecture, and testing and launching software products.
- Experience with C, High Performance Computing, Remote Direct Memory Access, Storage Systems, System Design, and Kernel Drivers.
Preferred qualifications:
- Master’s degree or PhD in Engineering, Computer Science, or a related technical field.
- 8 years of experience with data structures and algorithms.
- 5 years of experience in a technical leadership role leading project teams and setting technical direction.
- 3 years of experience working in a complex, matrixed organization involving cross-functional, or cross-business projects.
- Experience with Storage Systems or Machine Learning Infrastructure.
About the job
Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.
The High Performance Networking (HPN) team’s mission is to deliver scalable, state-of-the-art high-performance networking solutions to support a various range of high performance workloads for Google and Google Cloud .
The HPN team is at the core of Google's AI infrastructure. We design, deliver, and sustain Google's end-to-end RDMA stack, comprising:
Core IPU components and Linux kernel/Guest RDMA drivers.
Industry-leading acceleration technologies such as TPUDirect, TPUDirect Storage (TDS), and GPUDirect Storage (GDS).
As a Senior Staff Software Engineer, you will lead the evolution of this stack. You will drive the integration of the high-performance networking layers with industry-leading Parallel File Systems (PFS) and storage servers. Your technical leadership will directly shape the efficiency, scalability, and performance of Google's hyperscaler platform for next-generation AI/ML training and inference workloads.
The AI and Infrastructure team is redefining what’s possible. We empower Google customers with breakthrough capabilities and insights by delivering AI and Infrastructure at unparalleled scale, efficiency, reliability and velocity. Our customers include Googlers, Google Cloud customers, and billions of Google users worldwide.
We're the driving force behind Google's groundbreaking innovations, empowering the development of our cutting-edge AI models, delivering unparalleled computing power to global services, and providing the essential platforms that enable developers to build the future. From software to hardware our teams are shaping the future of world-leading hyperscale computing, with key teams working on the development of our TPUs, Vertex AI for Google Cloud, Google Global Networking, Data Center operations, systems research, and much more.
Individual pay is determined by factors including job-related skills, experience, and relevant education or training.US: $262000 - $364000 (USD) + 25% bonus target + equity + benefits
Learn more about benefits at Google.
Responsibilities
- Technical Leadership and Mentorship: Lead and coach a distributed engineering team, fostering innovation while aligning cross-functional groups on goals, priorities, and deliverables.
- Large-Scale and LLM Architecture: Design and deploy scalable software and advanced serving architectures, optimizing the LLM serving stack for efficiency and reliability.
- Performance and Network Engineering: Architect performant network transport across various NICs and platforms, tuning the communication stack to accelerate HPC, RDMA, and ML workloads.
- Strategic Roadmap and Trends: Partner with cross-organizational leaders to co-develop roadmaps bridging RDMA, storage, and AI/ML, staying ahead of training and inference advancements.
- Collaborative System Integration: Collaborate with hardware, software, and systems engineers to bridge communication gaps and ensure seamless hardware-software integration.
Note: By applying to this position you will have an opportunity to share your preferred working location from the following: Sunnyvale, CA, USA; New York, NY, USA.
Minimum qualifications:
- Bachelor's degree or equivalent practical experience.
- 8 years of experience programming in C++.
- 5 years of experience with design and architecture, and testing and launching software products.
- Experience with C, High Performance Computing, Remote Direct Memory Access, Storage Systems, System Design, and Kernel Drivers.
Preferred qualifications:
- Master’s degree or PhD in Engineering, Computer Science, or a related technical field.
- 8 years of experience with data structures and algorithms.
- 5 years of experience in a technical leadership role leading project teams and setting technical direction.
- 3 years of experience working in a complex, matrixed organization involving cross-functional, or cross-business projects.
- Experience with Storage Systems or Machine Learning Infrastructure.
About the job
Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.
The High Performance Networking (HPN) team’s mission is to deliver scalable, state-of-the-art high-performance networking solutions to support a various range of high performance workloads for Google and Google Cloud .
The HPN team is at the core of Google's AI infrastructure. We design, deliver, and sustain Google's end-to-end RDMA stack, comprising:
Core IPU components and Linux kernel/Guest RDMA drivers.
Industry-leading acceleration technologies such as TPUDirect, TPUDirect Storage (TDS), and GPUDirect Storage (GDS).
As a Senior Staff Software Engineer, you will lead the evolution of this stack. You will drive the integration of the high-performance networking layers with industry-leading Parallel File Systems (PFS) and storage servers. Your technical leadership will directly shape the efficiency, scalability, and performance of Google's hyperscaler platform for next-generation AI/ML training and inference workloads.
The AI and Infrastructure team is redefining what’s possible. We empower Google customers with breakthrough capabilities and insights by delivering AI and Infrastructure at unparalleled scale, efficiency, reliability and velocity. Our customers include Googlers, Google Cloud customers, and billions of Google users worldwide.
We're the driving force behind Google's groundbreaking innovations, empowering the development of our cutting-edge AI models, delivering unparalleled computing power to global services, and providing the essential platforms that enable developers to build the future. From software to hardware our teams are shaping the future of world-leading hyperscale computing, with key teams working on the development of our TPUs, Vertex AI for Google Cloud, Google Global Networking, Data Center operations, systems research, and much more.
Individual pay is determined by factors including job-related skills, experience, and relevant education or training.US: $262000 - $364000 (USD) + 25% bonus target + equity + benefits
Learn more about benefits at Google.
Responsibilities
- Technical Leadership and Mentorship: Lead and coach a distributed engineering team, fostering innovation while aligning cross-functional groups on goals, priorities, and deliverables.
- Large-Scale and LLM Architecture: Design and deploy scalable software and advanced serving architectures, optimizing the LLM serving stack for efficiency and reliability.
- Performance and Network Engineering: Architect performant network transport across various NICs and platforms, tuning the communication stack to accelerate HPC, RDMA, and ML workloads.
- Strategic Roadmap and Trends: Partner with cross-organizational leaders to co-develop roadmaps bridging RDMA, storage, and AI/ML, staying ahead of training and inference advancements.
- Collaborative System Integration: Collaborate with hardware, software, and systems engineers to bridge communication gaps and ensure seamless hardware-software integration.
See All 44 Infrastructure Software Engineer Jobs in New York
Find roles in New York that match your experience and apply in just a few clicks.
Find JobsInfrastructure Software Engineer Job Market in New York
Who's Hiring
Top Industries Hiring
- Technology & Software
- Science & Research
- Biotechnology & Pharmaceuticals
Infrastructure Software Engineer Jobs in New York: Frequently Asked Questions
How do I get a infrastructure software engineer job in New York?
New York's infrastructure engineering market is strongest in financial services, media and entertainment technology, and large enterprise cloud operations, with hiring concentrated in Midtown, Hudson Yards, and the Financial District. Candidates who stand out have hands-on experience with distributed systems, Kubernetes, or Terraform, and can point to production-scale work. Networking through New York tech meetups and targeting firms with large internal platform teams also gives a clear edge over remote applicants.
Which companies hire infrastructure software engineers in New York?
Employers hiring infrastructure software engineers in New York right now include Google, Meta, and Bloomberg, based on current listings on Migrate Mate as of November 2026. New York's hiring base is unusually broad, mixing global investment banks and trading firms, major media conglomerates, and high-growth tech companies with large engineering offices in the city.
Are there remote infrastructure software engineer jobs in New York?
Yes, though availability depends heavily on the role, since hands-on infrastructure work involving on-premises hardware or data center access is rarely fully remote. About 79% of infrastructure software engineer openings tied to New York are remote or hybrid as of November 2026, reflecting demand from financial and media firms that have embraced flexible arrangements for cloud and automation-focused roles.
How can I get a infrastructure software engineer job in New York with little or no experience?
The most realistic entry path in New York is through junior site reliability engineering or platform engineering roles at mid-size tech firms or fintech startups, many of which are concentrated in the Flatiron District and Lower Manhattan. Bootcamp graduates and candidates from CUNY or NYU who can demonstrate container orchestration projects or scripting skills get traction at companies scaling their internal infrastructure without large senior teams.
Which industries hire the most infrastructure software engineers in New York?
Most infrastructure software engineer openings in New York sit in Technology & Software, Science & Research, and Biotechnology & Pharmaceuticals, per current listings on Migrate Mate as of November 2026. New York's density of global financial institutions, streaming and media technology companies, and large enterprise software operations makes it one of the few cities where infrastructure engineers have genuine cross-industry options without leaving the metro area.
Related Jobs in New York
See All 44 Infrastructure Software Engineer Jobs in New York
Find roles in New York that match your experience and apply in just a few clicks.
Find Jobs