Infrastructure Software Engineer Jobs in New York
Infrastructure Software Engineer jobs in New York are among the most active in the country, concentrated in financial services, media and entertainment, healthcare technology, and cloud infrastructure across firms ranging from early-career hires to senior architects. The densest hiring clusters around New York City, with additional demand in Buffalo and Albany, where employers like JPMorgan Chase, IBM, and Verizon maintain significant engineering operations. The most sought-after specializations include Kubernetes and container orchestration, site reliability engineering, and distributed systems design. Find a role that fits below and apply directly.
Find JobsOverview
Showing 4 of 44+ Infrastructure Software Engineer jobs
Note: By applying to this position you will have an opportunity to share your preferred working location from the following: Sunnyvale, CA, USA; New York, NY, USA.
Minimum qualifications:
- Bachelor's degree or equivalent practical experience.
- 8 years of experience programming in C++.
- 5 years of experience with design and architecture, and testing and launching software products.
- Experience with C, High Performance Computing, Remote Direct Memory Access, Storage Systems, System Design, and Kernel Drivers.
Preferred qualifications:
- Master’s degree or PhD in Engineering, Computer Science, or a related technical field.
- 8 years of experience with data structures and algorithms.
- 5 years of experience in a technical leadership role leading project teams and setting technical direction.
- 3 years of experience working in a complex, matrixed organization involving cross-functional, or cross-business projects.
- Experience with Storage Systems or Machine Learning Infrastructure.
About the job
Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.
The High Performance Networking (HPN) team’s mission is to deliver scalable, state-of-the-art high-performance networking solutions to support a various range of high performance workloads for Google and Google Cloud .
The HPN team is at the core of Google's AI infrastructure. We design, deliver, and sustain Google's end-to-end RDMA stack, comprising:
Core IPU components and Linux kernel/Guest RDMA drivers.
Industry-leading acceleration technologies such as TPUDirect, TPUDirect Storage (TDS), and GPUDirect Storage (GDS).
As a Senior Staff Software Engineer, you will lead the evolution of this stack. You will drive the integration of the high-performance networking layers with industry-leading Parallel File Systems (PFS) and storage servers. Your technical leadership will directly shape the efficiency, scalability, and performance of Google's hyperscaler platform for next-generation AI/ML training and inference workloads.
The AI and Infrastructure team is redefining what’s possible. We empower Google customers with breakthrough capabilities and insights by delivering AI and Infrastructure at unparalleled scale, efficiency, reliability and velocity. Our customers include Googlers, Google Cloud customers, and billions of Google users worldwide.
We're the driving force behind Google's groundbreaking innovations, empowering the development of our cutting-edge AI models, delivering unparalleled computing power to global services, and providing the essential platforms that enable developers to build the future. From software to hardware our teams are shaping the future of world-leading hyperscale computing, with key teams working on the development of our TPUs, Vertex AI for Google Cloud, Google Global Networking, Data Center operations, systems research, and much more.
Individual pay is determined by factors including job-related skills, experience, and relevant education or training.US: $262000 - $364000 (USD) + 25% bonus target + equity + benefits
Learn more about benefits at Google.
Responsibilities
- Technical Leadership and Mentorship: Lead and coach a distributed engineering team, fostering innovation while aligning cross-functional groups on goals, priorities, and deliverables.
- Large-Scale and LLM Architecture: Design and deploy scalable software and advanced serving architectures, optimizing the LLM serving stack for efficiency and reliability.
- Performance and Network Engineering: Architect performant network transport across various NICs and platforms, tuning the communication stack to accelerate HPC, RDMA, and ML workloads.
- Strategic Roadmap and Trends: Partner with cross-organizational leaders to co-develop roadmaps bridging RDMA, storage, and AI/ML, staying ahead of training and inference advancements.
- Collaborative System Integration: Collaborate with hardware, software, and systems engineers to bridge communication gaps and ensure seamless hardware-software integration.
Note: By applying to this position you will have an opportunity to share your preferred working location from the following: Sunnyvale, CA, USA; New York, NY, USA.
Minimum qualifications:
- Bachelor's degree or equivalent practical experience.
- 8 years of experience programming in C++.
- 5 years of experience with design and architecture, and testing and launching software products.
- Experience with C, High Performance Computing, Remote Direct Memory Access, Storage Systems, System Design, and Kernel Drivers.
Preferred qualifications:
- Master’s degree or PhD in Engineering, Computer Science, or a related technical field.
- 8 years of experience with data structures and algorithms.
- 5 years of experience in a technical leadership role leading project teams and setting technical direction.
- 3 years of experience working in a complex, matrixed organization involving cross-functional, or cross-business projects.
- Experience with Storage Systems or Machine Learning Infrastructure.
About the job
Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.
The High Performance Networking (HPN) team’s mission is to deliver scalable, state-of-the-art high-performance networking solutions to support a various range of high performance workloads for Google and Google Cloud .
The HPN team is at the core of Google's AI infrastructure. We design, deliver, and sustain Google's end-to-end RDMA stack, comprising:
Core IPU components and Linux kernel/Guest RDMA drivers.
Industry-leading acceleration technologies such as TPUDirect, TPUDirect Storage (TDS), and GPUDirect Storage (GDS).
As a Senior Staff Software Engineer, you will lead the evolution of this stack. You will drive the integration of the high-performance networking layers with industry-leading Parallel File Systems (PFS) and storage servers. Your technical leadership will directly shape the efficiency, scalability, and performance of Google's hyperscaler platform for next-generation AI/ML training and inference workloads.
The AI and Infrastructure team is redefining what’s possible. We empower Google customers with breakthrough capabilities and insights by delivering AI and Infrastructure at unparalleled scale, efficiency, reliability and velocity. Our customers include Googlers, Google Cloud customers, and billions of Google users worldwide.
We're the driving force behind Google's groundbreaking innovations, empowering the development of our cutting-edge AI models, delivering unparalleled computing power to global services, and providing the essential platforms that enable developers to build the future. From software to hardware our teams are shaping the future of world-leading hyperscale computing, with key teams working on the development of our TPUs, Vertex AI for Google Cloud, Google Global Networking, Data Center operations, systems research, and much more.
Individual pay is determined by factors including job-related skills, experience, and relevant education or training.US: $262000 - $364000 (USD) + 25% bonus target + equity + benefits
Learn more about benefits at Google.
Responsibilities
- Technical Leadership and Mentorship: Lead and coach a distributed engineering team, fostering innovation while aligning cross-functional groups on goals, priorities, and deliverables.
- Large-Scale and LLM Architecture: Design and deploy scalable software and advanced serving architectures, optimizing the LLM serving stack for efficiency and reliability.
- Performance and Network Engineering: Architect performant network transport across various NICs and platforms, tuning the communication stack to accelerate HPC, RDMA, and ML workloads.
- Strategic Roadmap and Trends: Partner with cross-organizational leaders to co-develop roadmaps bridging RDMA, storage, and AI/ML, staying ahead of training and inference advancements.
- Collaborative System Integration: Collaborate with hardware, software, and systems engineers to bridge communication gaps and ensure seamless hardware-software integration.
See All 44 Infrastructure Software Engineer Jobs in New York
Find roles in New York that match your experience and apply in just a few clicks.
Find JobsInfrastructure Software Engineer Jobs by City in New York
Where New York roles are concentrated, by current openings.
Infrastructure Software Engineer Job Market in New York
A snapshot from current New York openings, updated as new roles post.
Who's Hiring
Top Industries Hiring
- Technology & Software
- Science & Research
- Biotechnology & Pharmaceuticals
What New York Employers Look For
The qualifications that appear most often in infrastructure software engineer jobs across New York.
- Bachelor's degree in computer science, engineering, or a closely related technical field
- Proficiency with cloud platforms such as AWS, Azure, or Google Cloud
- Hands-on experience with infrastructure-as-code tools like Terraform or Ansible
- Strong knowledge of Linux systems administration and networking fundamentals
- Experience designing and operating CI/CD pipelines in production environments
- Familiarity with containerization and orchestration platforms, particularly Kubernetes and Docker
Infrastructure Software Engineer Jobs in New York: Frequently Asked Questions
How do you become a infrastructure software engineer in New York?
Infrastructure software engineering in New York has no state-issued license requirement, so the hiring path centers on education and demonstrated technical skill. Most candidates enter with a bachelor's degree in computer science or a related field, though strong candidates also come from bootcamps paired with hands-on project portfolios. New York employers, particularly in finance and media, place significant weight on cloud certifications from AWS, Google Cloud, or Microsoft Azure as concrete proof of readiness.
How much do infrastructure software engineers make in New York?
Infrastructure software engineers in New York earn a median of about $166,180 a year, based on May 2025 Bureau of Labor Statistics wage data, ranging from around $97,430 for the lowest 10% to over $224,590 for the top 10%. Pay rises with experience, specialty, and employer.
Which companies hire infrastructure software engineers in New York?
Employers hiring infrastructure software engineers in New York right now include Google, Meta, and Bloomberg, based on current listings on Migrate Mate as of November 2026. New York's financial services and media sectors generate particularly consistent demand, with large institutions and tech-forward firms maintaining dedicated infrastructure and platform engineering teams year-round.
Which New York cities have the most infrastructure software engineer jobs?
New York and New York City account for the largest share of infrastructure software engineer openings in New York. New York City dominates by volume given its density of financial institutions, media companies, and enterprise technology firms, while Buffalo's growing presence of regional bank technology centers and Albany's state government IT operations contribute meaningful hiring outside the metro.
Are there remote infrastructure software engineer jobs in New York?
Yes, and more than most technical fields. About 69% of infrastructure software engineer openings tied to New York are remote or hybrid as of November 2026, reflecting how much of the work involves managing cloud environments and pipelines rather than physical hardware. Roles focused on cloud architecture, site reliability, and DevOps tooling are the most consistently offered in fully remote arrangements.
How can I get hired as a infrastructure software engineer in New York with little or no experience?
The most realistic entry path is through an associate or junior site reliability engineer role, which large New York financial firms and tech companies use to bring in candidates who have foundational Linux and scripting skills but lack production experience. IBM and Verizon in New York run structured rotational and graduate programs that place early-career engineers in infrastructure teams. Building a public portfolio using Terraform, GitHub Actions, or a self-hosted Kubernetes cluster on a personal project gives candidates a concrete edge when competing without a formal work history.
Where can I find and apply to infrastructure software engineer jobs in New York?
You can find and apply to infrastructure software engineer jobs in New York on Migrate Mate, which lists current New York openings updated regularly. Search the available roles, find the ones that match your background and preferred location, and apply directly to the employers posting them.
See All 44 Infrastructure Software Engineer Jobs in New York
Find roles in New York that match your experience and apply in just a few clicks.
Find Jobs