ML Software Engineer Jobs in New York, NY
ML Software Engineer jobs in New York are in strong demand, concentrated in Midtown Manhattan, the Flatiron District, and Hudson Yards across finance, media technology, and enterprise software. Employers actively hiring right now include Meta, Google, and Tubi. See the openings below and apply to the ones that match your experience.
Find ML Software Engineer JobsOverview
Showing 5 of 12+ ML Software Engineer jobs
Minimum qualifications:
- Bachelor's degree or equivalent practical experience.
- 8 years of experience programming in C++ or Python.
- 5 years of experience testing, and launching software products.
- 5 years of experience with performance, large-scale systems data analysis, visualization tools, or debugging.
- 3 years of experience with software design and architecture.
Preferred qualifications:
- Experience with state-of-the-art ML compilers and their internals, experience writing compiler optimization passes.
- Experience with debugging correctness and performance issues at all levels of the ML software stack.
- Familiarity with accelerator HW architectures (TPUs/GPUs).
About the job
Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.
With your technical expertise you will manage project priorities, deadlines, and deliverables. You will design, develop, test, deploy, maintain, and enhance software solutions.
As a part of this team, you will build the Accelerated Linear Algebra (XLA) compiler which enables Tensor Processing Unit (TPUs), Google's in-house custom designed processor, to accelerate machine learning and other scientific computing workloads for both internal Google customers and external Cloud customers.
You will need to support new workloads, optimize for new models and new characteristics, as well as support new TPU hardware across multiple generations.
In this role, you will be working on a state-of-the-art TPU compiler with opportunities to work up and down the compiler stack, as well as on end user ML models and on Hardware (HW)/Software (SW) co-design.
The AI and Infrastructure team is redefining what’s possible. We empower Google customers with breakthrough capabilities and insights by delivering AI and Infrastructure at unparalleled scale, efficiency, reliability and velocity. Our customers include Googlers, Google Cloud customers, and billions of Google users worldwide.
We're the driving force behind Google's groundbreaking innovations, empowering the development of our cutting-edge AI models, delivering unparalleled computing power to global services, and providing the essential platforms that enable developers to build the future. From software to hardware our teams are shaping the future of world-leading hyperscale computing, with key teams working on the development of our TPUs, Vertex AI for Google Cloud, Google Global Networking, Data Center operations, systems research, and much more.
Individual pay is determined by factors including job-related skills, experience, and relevant education or training.US: $207000 - $300000 (USD) + 20% bonus target + equity + benefits
Learn more about benefits at Google.
Responsibilities
- Contribute to the compiler for a novel processor designed to accelerate machine learning workloads.
- Target and compile high-performance implementations of operations at distributed scale.
- Design and implement new compiler passes that extract more performance out of current and next-generation TPUs, directly impacting fleet efficiency.
- Collaborate closely with hardware designers to co-design future processors.
- Research high-level representations to effectively program large-scale, distributed, and heterogeneous systems.
Minimum qualifications:
- Bachelor's degree or equivalent practical experience.
- 8 years of experience programming in C++ or Python.
- 5 years of experience testing, and launching software products.
- 5 years of experience with performance, large-scale systems data analysis, visualization tools, or debugging.
- 3 years of experience with software design and architecture.
Preferred qualifications:
- Experience with state-of-the-art ML compilers and their internals, experience writing compiler optimization passes.
- Experience with debugging correctness and performance issues at all levels of the ML software stack.
- Familiarity with accelerator HW architectures (TPUs/GPUs).
About the job
Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.
With your technical expertise you will manage project priorities, deadlines, and deliverables. You will design, develop, test, deploy, maintain, and enhance software solutions.
As a part of this team, you will build the Accelerated Linear Algebra (XLA) compiler which enables Tensor Processing Unit (TPUs), Google's in-house custom designed processor, to accelerate machine learning and other scientific computing workloads for both internal Google customers and external Cloud customers.
You will need to support new workloads, optimize for new models and new characteristics, as well as support new TPU hardware across multiple generations.
In this role, you will be working on a state-of-the-art TPU compiler with opportunities to work up and down the compiler stack, as well as on end user ML models and on Hardware (HW)/Software (SW) co-design.
The AI and Infrastructure team is redefining what’s possible. We empower Google customers with breakthrough capabilities and insights by delivering AI and Infrastructure at unparalleled scale, efficiency, reliability and velocity. Our customers include Googlers, Google Cloud customers, and billions of Google users worldwide.
We're the driving force behind Google's groundbreaking innovations, empowering the development of our cutting-edge AI models, delivering unparalleled computing power to global services, and providing the essential platforms that enable developers to build the future. From software to hardware our teams are shaping the future of world-leading hyperscale computing, with key teams working on the development of our TPUs, Vertex AI for Google Cloud, Google Global Networking, Data Center operations, systems research, and much more.
Individual pay is determined by factors including job-related skills, experience, and relevant education or training.US: $207000 - $300000 (USD) + 20% bonus target + equity + benefits
Learn more about benefits at Google.
Responsibilities
- Contribute to the compiler for a novel processor designed to accelerate machine learning workloads.
- Target and compile high-performance implementations of operations at distributed scale.
- Design and implement new compiler passes that extract more performance out of current and next-generation TPUs, directly impacting fleet efficiency.
- Collaborate closely with hardware designers to co-design future processors.
- Research high-level representations to effectively program large-scale, distributed, and heterogeneous systems.
See All 12 ML Software Engineer Jobs in New York
Find roles in New York that match your experience and apply in just a few clicks.
Find ML Software Engineer JobsML Software Engineer Job Market in New York
Who's Hiring
Top Industries Hiring
- Technology & Software
- Media & Entertainment
ML Software Engineer Jobs in New York: Frequently Asked Questions
How do I get a ml software engineer job in New York?
Target New York's densest hiring pockets: financial services firms in Midtown and Lower Manhattan, media and advertising technology companies in the Flatiron District, and enterprise software companies in Hudson Yards. Candidates who can demonstrate experience with production ML systems, not just research notebooks, stand out here. Networking through New York-based ML meetups and applied AI communities gives an additional edge over remote applicants.
Which companies hire ml software engineers in New York?
New York ml software engineer roles are posted by Meta, Google, and Tubi and others right now, based on current listings on Migrate Mate as of August 2026. New York's mix of Wall Street firms, global media companies, and fast-growing fintech and adtech startups means hiring comes from a broader range of industries than most U.S. cities.
Are there remote ml software engineer jobs in New York?
Yes, ml software engineer work is well-suited to remote arrangements since the core work involves code, models, and data rather than physical presence. About 33% of ml software engineer openings tied to New York are remote or hybrid as of August 2026, with hybrid schedules most common at larger financial and enterprise employers. Modeling, experimentation, and pipeline work are the functions most frequently offered remotely.
How can I get a ml software engineer job in New York with little or no experience?
The most realistic entry path in New York is targeting mid-size fintech or adtech startups in Brooklyn Tech Triangle or Midtown South, which frequently hire junior ML engineers willing to work across the full stack. Entry-level data engineer and ML platform engineer roles at these companies often transition into dedicated ML engineering positions within a year. Building a portfolio of deployed projects, even small ones hosted publicly, consistently separates entry-level candidates in New York's competitive applicant pool.
Which industries hire the most ml software engineers in New York?
New York ml software engineer roles concentrate in Technology & Software and Media & Entertainment, based on current listings on Migrate Mate as of August 2026. New York's outsized presence in financial services, digital advertising, and media technology creates sustained demand for applied ML engineering that distinguishes the city from tech hubs focused primarily on consumer software.
Related Jobs in New York
See All 12 ML Software Engineer Jobs in New York
Find roles in New York that match your experience and apply in just a few clicks.
Find ML Software Engineer Jobs