ML Engineer Jobs in Virginia

Brief Role Description

This position is Career Level 29P and located in Reston, VA, with a Role Classification of Hybrid.

Base pay is one part of our total compensation package and is established within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay for this position will fall within the pay range listed below and is determined based on job-related factors including, but not limited to, skills, experience, qualifications, work location, and market conditions. Base salary is localized according to an employee’s work location. Ranges are market-dependent and may be modified in the future.

This position is located in Reston, VA. The pay range for this location is $146,700 - $264,100 annually.

Benefits for this position include:

• Eligibility for annual performance bonus
• Healthcare benefits
• 401(k), with company match
• Defined contribution retirement program
• Tuition reimbursement
• Company lease car program
• Paid time off

Role Summary

The Lead MLOps Engineer is the senior‑most individual contributor responsible for the enterprise AI platform at VWFS and serves as the technical authority for how AI models are built, operationalized, governed, and run at scale. Reporting to the Data and AI Platform team, this role defines the multi‑year technical vision, reference architecture, and standards for the AWS‑based MLOps platform supporting classical machine learning, Generative AI, and agentic AI workloads—while remaining actively hands‑on in its implementation.

This role is explicitly forward‑looking, shaping VWFS’s long‑term AI platform roadmap by evaluating emerging architectures and vendors, and is accountable for platform scalability, security, observability, and regulatory compliance. The Lead MLOps Engineer regularly contributes production‑quality code, building and debugging pipelines, infrastructure, and platform components to validate designs and accelerate adoption.
Impact is delivered through architectural ownership combined with sustained hands‑on contribution—driving executive investment decisions, owning enterprise MLOps standards and golden paths, and acting as a senior technical thought partner across Data, AI, Application Development, Risk, and Governance—leading by technical example rather than formal people management.

Responsibilities within this Role

• Own and drive the multi‑year technical strategy and hands‑on implementation of VCI’s AWS‑based MLOps platform, actively building and evolving core platform components that support classical ML, Generative AI, and agentic AI workloads at enterprise scale (35%).
• Lead horizon‑scanning and practical validation across the MLOps, GenAI, and agentic AI ecosystem—evaluating architectures, frameworks, and vendors, and prototyping or implementing key patterns to inform platform decisions (e.g., model registry, feature store, observability, vector databases, agent orchestration) and the AI platform roadmap (20%).
• Design, implement, and continuously refine enterprise‑scale CI/CD, deployment, monitoring, and rollback patterns for ML and GenAI workloads. Actively write and maintain production code for pipelines, automation, and platform tooling, while authoring design artifacts and technical guidance that reinforce reliable, scalable operations (15%).
• Define, steward, and evolve enterprise MLOps standards, reference architectures, and golden paths, ensuring adoption through working implementations, reference code, and direct contribution across the AI CoE, Data Engineering & Analytics, and other Business Units (10%).
• Serve as the senior hands‑on technical leader for the MLOps discipline—leading design reviews, pairing on complex problems, and raising engineering and operational quality by coding alongside teams, driving cross‑organizational alignment through influence rather than line management (10%).
• Partner with AI Governance, GRC, and Risk to design and operationalize embedded controls for model risk management, security, auditability, and fair lending, implementing controls directly in pipelines and platform code to ensure compliance is enforced by design (10%).

Qualification requirements

Relevant Experience:

• 12–15+ years of overall experience in software engineering, data engineering, or machine learning engineering
• 7+ years of hands-on specifically in MLOps, ML platform engineering, or productionizing machine learning systems at enterprise scale.
• 5-7 years utilizing methods and tools to extract, curate, explore, and analyze data from large data sources
• Track record of operating as the top technical individual contributor on an ML platform — setting multi-year technical strategy, anticipating where the discipline is heading, owning architectural decisions across classical ML and generative / agentic AI, and influencing decisions on build-versus-buy, and vendor selection.
• Extensive experience leading the design, deploy and operation of end-to-end ML platform as well as scaling the same on AWS
• Proven technical leadership establishing enterprise-wide MLOps standards and architectures
• Deep understanding of ML model lifecycle, deployment patterns, and serving frameworks

Required Education:

• Master’s degree in computer science, Data Science, Engineering, Statistics, Operations Research or similar quantitative STEM field

Skills:

• Strong strategic and systems-thinking mindset, with the ability to connect technical decisions to long-horizon business outcomes.
• Ability to translate complex technical concepts for diverse stakeholders, from engineers to senior executives.
• Proven influence without formal authority across engineering, data science, AI, and governance teams.
• Continuous improvement mindset with a passion for operational excellence, reliability, and platform craftsmanship.
• Innate curiosity and collaborative style, partners with different business units to co-create solutions rather than impose them.
• A fantastic storyteller, at ease presenting to and able to influence senior executives.

Required Skills:

• Hands‑on AWS MLOps engineering: Actively builds and debugs production systems on SageMaker, EKS/ECS, EC2, S3, Lake Formation, and Lambda—regularly contributing code, not just designs.
• Expert Python engineering: Writes production‑quality Python routinely for ML pipelines, automation, and platform tooling; deep experience with Docker, Kubernetes/EKS, and CloudFormation/Terraform.
• CI/CD for ML (implemented): Designs and codes CI/CD pipelines for ML, including testing, deployments, rollbacks, and progressive releases.
• Workflow orchestration: Builds and maintains complex workflows using Apache Airflow, writing DAGs and custom operators.
• Platform components (hands‑on): Contributes code to feature stores, experiment tracking, and model registries used across multiple business units.
• Governance by implementation: Implements integrations with Informatica IDMC and AWS Lake Formation, including metadata, access controls, and audit workflows.
• Production problem‑solving: Diagnoses and fixes issues in large‑scale, regulated ML platforms; comfortable diving into code, logs, and infrastructure.
• ML observability: Builds monitoring, drift detection, and performance optimization solutions end‑to‑end.
• Execution‑oriented influence: Leads by example—codes alongside teams, ships solutions, and drives change through working implementations.

#LI-MT1

#LI-Hybrid