ML Software Engineer Jobs in New York
ML Software Engineer jobs in New York are among the most active in the country, concentrated in finance, media, advertising technology, and enterprise software across experience levels from entry-level associate to principal engineer. The heaviest hiring happens in New York City, with additional demand in Buffalo and Albany tied to healthcare systems and state government modernization efforts. IBM, JPMorgan Chase, and Bloomberg are among the most consistent large employers of ml software engineers in New York, with strong demand in natural language processing, recommendation systems, and MLOps. Find a role that fits below and apply directly.
Find ML Software Engineer JobsOverview
Showing 5 of 12+ ML Software Engineer jobs
Minimum qualifications:
- Bachelor's degree or equivalent practical experience.
- 8 years of experience programming in C++ or Python.
- 5 years of experience testing, and launching software products.
- 5 years of experience with performance, large-scale systems data analysis, visualization tools, or debugging.
- 3 years of experience with software design and architecture.
Preferred qualifications:
- Experience with state-of-the-art ML compilers and their internals, experience writing compiler optimization passes.
- Experience with debugging correctness and performance issues at all levels of the ML software stack.
- Familiarity with accelerator HW architectures (TPUs/GPUs).
About the job
Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.
With your technical expertise you will manage project priorities, deadlines, and deliverables. You will design, develop, test, deploy, maintain, and enhance software solutions.
As a part of this team, you will build the Accelerated Linear Algebra (XLA) compiler which enables Tensor Processing Unit (TPUs), Google's in-house custom designed processor, to accelerate machine learning and other scientific computing workloads for both internal Google customers and external Cloud customers.
You will need to support new workloads, optimize for new models and new characteristics, as well as support new TPU hardware across multiple generations.
In this role, you will be working on a state-of-the-art TPU compiler with opportunities to work up and down the compiler stack, as well as on end user ML models and on Hardware (HW)/Software (SW) co-design.
The AI and Infrastructure team is redefining what’s possible. We empower Google customers with breakthrough capabilities and insights by delivering AI and Infrastructure at unparalleled scale, efficiency, reliability and velocity. Our customers include Googlers, Google Cloud customers, and billions of Google users worldwide.
We're the driving force behind Google's groundbreaking innovations, empowering the development of our cutting-edge AI models, delivering unparalleled computing power to global services, and providing the essential platforms that enable developers to build the future. From software to hardware our teams are shaping the future of world-leading hyperscale computing, with key teams working on the development of our TPUs, Vertex AI for Google Cloud, Google Global Networking, Data Center operations, systems research, and much more.
Individual pay is determined by factors including job-related skills, experience, and relevant education or training.US: $207000 - $300000 (USD) + 20% bonus target + equity + benefits
Learn more about benefits at Google.
Responsibilities
- Contribute to the compiler for a novel processor designed to accelerate machine learning workloads.
- Target and compile high-performance implementations of operations at distributed scale.
- Design and implement new compiler passes that extract more performance out of current and next-generation TPUs, directly impacting fleet efficiency.
- Collaborate closely with hardware designers to co-design future processors.
- Research high-level representations to effectively program large-scale, distributed, and heterogeneous systems.
Minimum qualifications:
- Bachelor's degree or equivalent practical experience.
- 8 years of experience programming in C++ or Python.
- 5 years of experience testing, and launching software products.
- 5 years of experience with performance, large-scale systems data analysis, visualization tools, or debugging.
- 3 years of experience with software design and architecture.
Preferred qualifications:
- Experience with state-of-the-art ML compilers and their internals, experience writing compiler optimization passes.
- Experience with debugging correctness and performance issues at all levels of the ML software stack.
- Familiarity with accelerator HW architectures (TPUs/GPUs).
About the job
Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.
With your technical expertise you will manage project priorities, deadlines, and deliverables. You will design, develop, test, deploy, maintain, and enhance software solutions.
As a part of this team, you will build the Accelerated Linear Algebra (XLA) compiler which enables Tensor Processing Unit (TPUs), Google's in-house custom designed processor, to accelerate machine learning and other scientific computing workloads for both internal Google customers and external Cloud customers.
You will need to support new workloads, optimize for new models and new characteristics, as well as support new TPU hardware across multiple generations.
In this role, you will be working on a state-of-the-art TPU compiler with opportunities to work up and down the compiler stack, as well as on end user ML models and on Hardware (HW)/Software (SW) co-design.
The AI and Infrastructure team is redefining what’s possible. We empower Google customers with breakthrough capabilities and insights by delivering AI and Infrastructure at unparalleled scale, efficiency, reliability and velocity. Our customers include Googlers, Google Cloud customers, and billions of Google users worldwide.
We're the driving force behind Google's groundbreaking innovations, empowering the development of our cutting-edge AI models, delivering unparalleled computing power to global services, and providing the essential platforms that enable developers to build the future. From software to hardware our teams are shaping the future of world-leading hyperscale computing, with key teams working on the development of our TPUs, Vertex AI for Google Cloud, Google Global Networking, Data Center operations, systems research, and much more.
Individual pay is determined by factors including job-related skills, experience, and relevant education or training.US: $207000 - $300000 (USD) + 20% bonus target + equity + benefits
Learn more about benefits at Google.
Responsibilities
- Contribute to the compiler for a novel processor designed to accelerate machine learning workloads.
- Target and compile high-performance implementations of operations at distributed scale.
- Design and implement new compiler passes that extract more performance out of current and next-generation TPUs, directly impacting fleet efficiency.
- Collaborate closely with hardware designers to co-design future processors.
- Research high-level representations to effectively program large-scale, distributed, and heterogeneous systems.
See All 12 ML Software Engineer Jobs in New York
Find roles in New York that match your experience and apply in just a few clicks.
Find ML Software Engineer JobsML Software Engineer Jobs by City in New York
Where New York roles are concentrated, by current openings.
ML Software Engineer Job Market in New York
A snapshot from current New York openings, updated as new roles post.
Who's Hiring
Top Industries Hiring
- Technology & Software
- Media & Entertainment
What New York Employers Look For
The qualifications that appear most often in ML software engineer jobs across New York.
- Bachelor's or master's degree in computer science, statistics, or a related quantitative field
- Proficiency in Python and core ML frameworks such as TensorFlow, PyTorch, or scikit-learn
- Experience designing, training, and deploying machine learning models in production environments
- Familiarity with cloud platforms including AWS, Google Cloud, or Azure for ML workloads
- Strong understanding of data pipelines, feature engineering, and model evaluation techniques
- Ability to collaborate cross-functionally with data engineers, product managers, and business stakeholders
ML Software Engineer Jobs in New York: Frequently Asked Questions
How do you become a ml software engineer in New York?
There is no state-issued license required to work as an ml software engineer in New York. The standard path is a bachelor's or master's degree in computer science, mathematics, or a related field, followed by demonstrated project experience through internships, open-source contributions, or a strong portfolio. New York employers in finance and media often prioritize candidates who can show end-to-end model deployment, not just academic coursework.
How much do ML software engineers make in New York?
ML software engineers in New York earn a median of about $166,180 a year, based on May 2025 Bureau of Labor Statistics wage data, ranging from around $97,430 for the lowest 10% to over $224,590 for the top 10%. Pay rises with experience, specialty, and employer.
Which companies hire ml software engineers in New York?
Employers hiring ml software engineers in New York right now include Meta, Google, and Tubi, based on current listings on Migrate Mate as of August 2026. New York's density of financial services firms, advertising technology companies, and large media organizations makes it one of the most diverse hiring markets for machine learning talent in the country.
Which New York cities have the most ml software engineer jobs?
New York have the most ml software engineer openings in New York. New York City drives the majority of demand through its concentration of investment banks, hedge funds, and tech companies, while Buffalo and Albany see steady openings tied to healthcare technology initiatives and state agency modernization projects that have drawn engineering teams outside the main metro.
Are there remote ml software engineer jobs in New York?
Yes, and more than most fields. About 33% of ml software engineer openings tied to New York are remote or hybrid as of August 2026, reflecting how well this work translates to distributed environments. Model development, experimentation, and code review are the parts of the role most commonly done fully remote, while on-site expectations tend to apply to embedded teams at trading firms or media companies with proprietary infrastructure.
How can I get hired as a ml software engineer in New York with little or no experience?
The most realistic entry path is a master's program with an industry capstone or internship at a New York employer, as firms like IBM and JPMorgan Chase run structured new-graduate programs that convert interns to full-time hires. Candidates transitioning from data analyst or software engineering roles can move laterally by completing a machine learning specialization and building a portfolio of deployed models. Contributing to open-source ML projects and targeting associate ML engineer postings at New York startups are also credible starting points.
Where can I find and apply to ml software engineer jobs in New York?
You can find and apply to ml software engineer jobs in New York on Migrate Mate, which lists current New York openings updated regularly. Search the listings, find roles that fit your experience and location, and apply directly to the employers posting them.
See All 12 ML Software Engineer Jobs in New York
Find roles in New York that match your experience and apply in just a few clicks.
Find ML Software Engineer Jobs