Scientist Jobs in Washington, DC

AI Data Scientist - TEVVAbout Bright Star

Bright Star is a forward-thinking technology consultancy that specializes in harnessing the power of data and AI to solve complex business challenges. Our team of experts combines deep technical expertise with strategic insight to deliver innovative solutions that drive meaningful impact for our clients.

Role Description

We are seeking a highly skilled AI Test, Evaluation, Verification & Validation (TEVV) Scientist to support a federal civilian agency's artificial intelligence governance and assurance initiatives. This role will be responsible for independently evaluating AI capabilities developed by government organizations, contractors, and commercial vendors to ensure they meet performance, reliability, security, and responsible AI standards.

The ideal candidate combines deep technical expertise in artificial intelligence with hands-on experience evaluating generative and agentic AI systems. This individual will assess AI performance, identify risks and failure modes, and provide actionable recommendations to support informed deployment decisions.

Key Responsibilities

· Plan and conduct independent tests, evaluation, verification, and validation (TEVV) activities for AI systems and solutions.

· Develop evaluation methods, test criteria, and acceptance thresholds

· Assess AI systems for accuracy, robustness, reliability, and operational effectiveness.

· Evaluate and document AI failure modes, hallucinations, bias, privacy leakage, prompt injection vulnerabilities, and model drift.

· Conduct assessments of generative AI, large language models (LLMs), agentic AI systems, and traditional machine learning solutions.

· Evaluate the effectiveness of human oversight mechanisms, governance controls, and safeguard implementations.

· Develop findings, risk assessments, severity ratings, and remediation recommendations for stakeholders.

· Prepare technical reports, executive summaries, and briefing materials that clearly communicate evaluation results and associated risks.

· Collaborate with technical, operational, and program stakeholders to improve AI system performance and trustworthiness.

Required Qualifications

· PhD in Computer Science, Data Science, Statistics, Artificial Intelligence, Machine Learning, or a related technical field with 8+ years of experience evaluating, testing, validating, or assessing AI/ML systems, including generative AI technologies

· Experience testing, evaluating, validating, or assessing AI and machine learning systems.

· Experience evaluating generative AI and agentic AI technologies, not solely traditional machine learning models.

· Experience in AI performance metrics, model validation techniques, and risk assessment methodologies.

· Experience identifying and analyzing AI vulnerabilities, failure modes, bias, privacy concerns, and security risks.

· Ability to communicate complex technical findings to both technical and non-technical audiences.

· Ability to obtain and maintain a Federal Public Trust clearance.

Preferred Qualifications

· Knowledge of NIST AI Risk Management Framework (AI RMF) and related AI assurance standards.

· Experience developing AI testing frameworks, red-teaming methodologies, or adversarial testing approaches

· Active Public Trust or other federal clearance.

Job Details

Job Type: Full-time

· Onsite in Washington DC

· Pay: $120k-$190k

Benefits:

· 401(k)

· 401(k) matching

· Dental insurance

· Health insurance

· Paid time off

· Vision insurance

Pay: $120,000.00 - $190,000.00 per year

Benefits:

• 401(k)
• 401(k) matching
• Dental insurance
• Health insurance
• Health savings account
• Paid time off
• Vision insurance

Application Question(s):

• Due to the nature of our work as a government contractor, all employees are required to have full U.S Citizenship at the time of hire. Are you a U.S Citizen?
• Do you have an active security clearance or public trust?
• Do you have a PhD in computer science, data science, statistics, artificial intelligence, machine learning, or a related technical field?
• Do you have 8 or more years of experience evaluating, testing, validating, or assessing AI/ML systems?
• Have you evaluated generative AI or agentic AI systems (for example, LLMs, chatbots, or AI agents), not only traditional machine learning models?
• Have you identified and documented AI failure modes such as hallucinations, bias, privacy leakage, prompt injection, or model drift?
• Are you able to work onsite in Washington, DC, full-time?

Work Location: In person