Big Data Engineer Jobs
Big Data Engineer jobs are open across technology, finance, healthcare, and retail, from entry-level to staff and principal, with specializations in distributed systems, data pipeline architecture, and real-time streaming. See the openings below and apply to the ones that match your experience.
Find Big Data Engineer JobsOverview
Showing 4 of 59+ Big Data Engineer jobs
About the Role
PubMatic is seeking for a passionate Data Engineer to join our Mobile Advertising Platform engineering team. In this role, you will build highly scalable data platforms powering Mobile Programmatic Advertising, RTB bidding systems, attribution, campaign optimization, fraud detection, and real-time analytics.
You will design and implement distributed systems capable of processing billions of mobile advertising events every day including ad requests, impressions, clicks, installs, conversions, in-app events, and auction logs. You'll build both real-time event processing and large-scale batch processing systems on PubMatic's in-house data centers, leveraging modern Big Data technologies (Spark, Kafka, Hadoop), high-performance services in Go and Rust, and Generative AI to deliver intelligent analytics and automation solutions.
You'll collaborate with Product Managers, ML Engineers, Data Scientists, and Backend Engineers to build next-generation Mobile AdTech products.
What You'll Do:
- Design and build highly scalable data platforms capable of processing billions of mobile advertising events with low latency and high availability.
- Design and build ultra-low-latency, high-throughput data pipelines processing the full mobile event funnel — ad request impression click install in-app conversion event — including OpenRTB bid request/response logging, auction analytics, and win-rate/bid-landscape reporting across multiple SSPs and ad exchanges, using Kafka, Spark Streaming, and Hadoop on our in-house data centers.
- Architect real-time campaign pacing and budget delivery systems, applying bid optimization techniques so spend controls and bid strategy adjustments react to live auction signals within seconds.
- Architect and implement real-time streaming pipelines to ingest, process, and analyze billions of advertising events including impressions, clicks, installs, conversions, and in-app events.
- Build real-time event processing systems with event-time semantics, windowed aggregations, watermarking, late/out-of-order event handling, deduplication, and exactly-once or idempotent delivery guarantees across Kafka and Spark Structured Streaming.
- Design and optimize scalable ETL/ELT pipelines and data models supporting Mobile AdTech use cases such as campaign reporting, attribution, audience segmentation, fraud detection, and revenue analytics.
- Develop and tune large-scale batch processing jobs on Spark and Hadoop (HDFS/YARN) — hourly and daily aggregations, backfills, reprocessing, and data reconciliation between real-time and batch layers — with strong workflow orchestration, SLAs, and data-quality checks.
- Design large-scale Invalid Traffic (IVT) and ad fraud detection systems — click spam, install hijacking, SDK spoofing, device farms — using rules-based and ML-driven detection across billions of daily events.
- Develop high-performance backend services and data processors (Java, Go, Rust, REST APIs, JDBC) supporting DSP bidding logic, campaign management tools, and reporting APIs.
- Operate and scale data infrastructure in PubMatic's in-house data centers — Kafka, Spark, and Hadoop clusters — including capacity planning, performance tuning, storage and compute efficiency, and cost optimization on bare-metal hardware.
- Design and develop GenAI-powered agents for analytics, operations, and data enrichment using frameworks such as LangChain, LlamaIndex, LangGraph, or custom orchestration frameworks.
- Design systems that are highly available, fault tolerant, and optimized for low latency and high throughput.
- Collaborate closely with cross-functional teams to improve platform scalability, reliability, performance, and customer experience.
- Work with Product Managers to define and implement new analytics capabilities and platform features.
- Participate in Agile/Scrum ceremonies including sprint planning, backlog grooming, estimation, retrospectives, and release planning.
- Perform architecture discussions, design reviews, and peer code reviews while promoting software engineering best practices.
We'd Love for You to Have:
- 1–5+ years of professional experience in Java backend development and data engineering.
- Strong computer science fundamentals including data structures, algorithms, distributed systems, and software design principles.
- Hands-on experience developing scalable backend services using Java, REST APIs, JDBC, and relational databases.
- Strong experience building distributed real-time and batch data pipelines using Spark, Kafka, Hadoop, and SQL on on-premise / in-house data center infrastructure.
- Experience designing and optimizing large-scale analytical data platforms for Mobile AdTech using Java, Go, Rust, Spark, Kafka, and SQL. Strong understanding of real-time event processing, distributed data systems, and data warehouse architectures. Familiarity with distributed query engines, modern data lake technologies, and high-performance analytical databases (e.g., Trino, Presto, ClickHouse, Apache Iceberg, Delta Lake, or Apache Hudi) is an added advantage.
- Experience processing high-volume streaming data using Spark Streaming, Kafka Streams, or similar technologies.
- Programming proficiency in Go and/or Rust for building high-throughput, low-latency data services, event collectors, and stream processors, in addition to Java.
- Solid understanding of real-time vs. batch processing trade-offs (Lambda/Kappa architectures), event-time processing, partitioning strategies, schema evolution (Avro/Protobuf/Parquet), and data correctness at scale.
- Good understanding of data warehousing, dimensional modeling, ETL/ELT design, and distributed computing concepts.
- Excellent verbal and written communication skills.
- Strong interpersonal skills with the ability to collaborate effectively across engineering, product, and customer-facing teams.
Bonus Qualifications
Mobile AdTech Experience
Experience working with Mobile AdTech platforms or digital advertising ecosystems is highly preferred, including knowledge of:
- Direct experience with Real-Time Bidding (RTB), OpenRTB protocol, and Mobile DSP/SSP/Ad Exchange architecture.
- Familiarity with mobile SDK event tracking (impressions, clicks, installs, in-app events) and Mobile Measurement Partner (MMP) integrations.
- Understanding of attribution methodologies (click-through, view-through, multi-touch-based attribution).
- Knowledge of mobile identity solutions (IDFA, GAID, IDFV) and the privacy landscape driving their deprecation.
- Experience with campaign pacing, bid optimization, and budget delivery algorithms.
- Background in ad fraud/IVT detection — click spam, install fraud, SDK spoofing.
Generative AI Experience
Experience building production-grade GenAI applications including:
- LLM integration (OpenAI, Anthropic, Claude, Gemini, Mistral, Cohere, etc.)
- LangChain, LlamaIndex, LangGraph, or similar orchestration frameworks
- Prompt engineering and prompt optimization
- AI evaluation and observability
- AI agents and workflow automation
Additional Preferred Skills
- Experience building scalable, production-grade AI or data platforms.
- Strong SQL optimization and query performance tuning skills.
- Experience running Big Data platforms on-premise — managing Kafka, Spark, and Hadoop/HDFS clusters, capacity planning, and hardware-aware performance tuning in a private data center environment.
- Experience with CI/CD pipelines, Docker, Kubernetes, and containerized application deployment on bare-metal / in-house data center infrastructure.
- Experience with monitoring and observability tools such as Grafana, Prometheus, Datadog, or similar.
- Strong debugging and troubleshooting skills in distributed environments.
- Demonstrated ability to achieve stretch goals in a fast-paced, highly collaborative engineering environment.
- Ability to learn new technologies quickly and independently.
Why Join Us?
- Work on high-scale data infrastructure processing billions of mobile advertising events every day, directly impacting publisher revenue, DSP spend, and advertiser outcomes across the in-app advertising ecosystem.
- Help shape the future of real-time analytics, attribution, campaign pacing, fraud/IVT detection, and GenAI-powered automation for mobile app monetization.
- Build modern data platforms with Spark, Kafka, Hadoop, Go, and Rust on PubMatic's in-house data centers at internet scale, with low latency and high availability.
- Collaborate with industry-leading engineers, product managers, data scientists, ML engineers, DSP partners, and publisher teams.
- Competitive compensation, performance-based incentives, and strong career growth opportunities in a globally recognized AdTech company.
AI-Enabled Engineering Mindset
We value engineers who actively leverage Generative AI tools and IDEs (e.g., GitHub Copilot, ChatGPT, Claude, Cursor, Windsurf) to accelerate development, improve code and data quality, automate repetitive tasks, and enhance documentation and debugging workflows. Engineers who demonstrate AI-first thinking — using these tools to drive faster experimentation, ideation, and technical execution — will thrive in our high-scale, data-intensive environment.
Qualification:
Should have a bachelor's degree in engineering or an equivalent degree from a well-known institute/university.
Additional Information:
Return to Office: PubMatic employees throughout the globe have returned to our offices via a hybrid work schedule (3 days "in office" and 2 days "working remotely") that is intended to maximize collaboration, innovation, and productivity among teams and across functions.
Benefits: Our benefits package includes the best of what leading organizations provide, such as paternity/maternity leave, healthcare insurance, broadband reimbursement. As well, when we're back in the office, we all benefit from a kitchen loaded with healthy snacks and drinks and catered lunches and much more!.
Diversity and Inclusion: PubMatic is proud to be an equal opportunity employer; we don't just value diversity, we promote and celebrate it. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status
About PubMatic
PubMatic is one of the world's leading scaled digital advertising platforms, offering more transparent advertising solutions to publishers, media buyers, commerce companies and data owners, allowing them to harness the power and potential of the open internet to drive better business outcomes.
Founded in 2006 with the vision that data-driven decisioning would be the future of digital advertising, we enable content creators to run a more profitable advertising business, which in turn allows them to invest back into the multi-screen and multi-format content that consumers demand.
About the Role
PubMatic is seeking for a passionate Data Engineer to join our Mobile Advertising Platform engineering team. In this role, you will build highly scalable data platforms powering Mobile Programmatic Advertising, RTB bidding systems, attribution, campaign optimization, fraud detection, and real-time analytics.
You will design and implement distributed systems capable of processing billions of mobile advertising events every day including ad requests, impressions, clicks, installs, conversions, in-app events, and auction logs. You'll build both real-time event processing and large-scale batch processing systems on PubMatic's in-house data centers, leveraging modern Big Data technologies (Spark, Kafka, Hadoop), high-performance services in Go and Rust, and Generative AI to deliver intelligent analytics and automation solutions.
You'll collaborate with Product Managers, ML Engineers, Data Scientists, and Backend Engineers to build next-generation Mobile AdTech products.
What You'll Do:
- Design and build highly scalable data platforms capable of processing billions of mobile advertising events with low latency and high availability.
- Design and build ultra-low-latency, high-throughput data pipelines processing the full mobile event funnel — ad request impression click install in-app conversion event — including OpenRTB bid request/response logging, auction analytics, and win-rate/bid-landscape reporting across multiple SSPs and ad exchanges, using Kafka, Spark Streaming, and Hadoop on our in-house data centers.
- Architect real-time campaign pacing and budget delivery systems, applying bid optimization techniques so spend controls and bid strategy adjustments react to live auction signals within seconds.
- Architect and implement real-time streaming pipelines to ingest, process, and analyze billions of advertising events including impressions, clicks, installs, conversions, and in-app events.
- Build real-time event processing systems with event-time semantics, windowed aggregations, watermarking, late/out-of-order event handling, deduplication, and exactly-once or idempotent delivery guarantees across Kafka and Spark Structured Streaming.
- Design and optimize scalable ETL/ELT pipelines and data models supporting Mobile AdTech use cases such as campaign reporting, attribution, audience segmentation, fraud detection, and revenue analytics.
- Develop and tune large-scale batch processing jobs on Spark and Hadoop (HDFS/YARN) — hourly and daily aggregations, backfills, reprocessing, and data reconciliation between real-time and batch layers — with strong workflow orchestration, SLAs, and data-quality checks.
- Design large-scale Invalid Traffic (IVT) and ad fraud detection systems — click spam, install hijacking, SDK spoofing, device farms — using rules-based and ML-driven detection across billions of daily events.
- Develop high-performance backend services and data processors (Java, Go, Rust, REST APIs, JDBC) supporting DSP bidding logic, campaign management tools, and reporting APIs.
- Operate and scale data infrastructure in PubMatic's in-house data centers — Kafka, Spark, and Hadoop clusters — including capacity planning, performance tuning, storage and compute efficiency, and cost optimization on bare-metal hardware.
- Design and develop GenAI-powered agents for analytics, operations, and data enrichment using frameworks such as LangChain, LlamaIndex, LangGraph, or custom orchestration frameworks.
- Design systems that are highly available, fault tolerant, and optimized for low latency and high throughput.
- Collaborate closely with cross-functional teams to improve platform scalability, reliability, performance, and customer experience.
- Work with Product Managers to define and implement new analytics capabilities and platform features.
- Participate in Agile/Scrum ceremonies including sprint planning, backlog grooming, estimation, retrospectives, and release planning.
- Perform architecture discussions, design reviews, and peer code reviews while promoting software engineering best practices.
We'd Love for You to Have:
- 1–5+ years of professional experience in Java backend development and data engineering.
- Strong computer science fundamentals including data structures, algorithms, distributed systems, and software design principles.
- Hands-on experience developing scalable backend services using Java, REST APIs, JDBC, and relational databases.
- Strong experience building distributed real-time and batch data pipelines using Spark, Kafka, Hadoop, and SQL on on-premise / in-house data center infrastructure.
- Experience designing and optimizing large-scale analytical data platforms for Mobile AdTech using Java, Go, Rust, Spark, Kafka, and SQL. Strong understanding of real-time event processing, distributed data systems, and data warehouse architectures. Familiarity with distributed query engines, modern data lake technologies, and high-performance analytical databases (e.g., Trino, Presto, ClickHouse, Apache Iceberg, Delta Lake, or Apache Hudi) is an added advantage.
- Experience processing high-volume streaming data using Spark Streaming, Kafka Streams, or similar technologies.
- Programming proficiency in Go and/or Rust for building high-throughput, low-latency data services, event collectors, and stream processors, in addition to Java.
- Solid understanding of real-time vs. batch processing trade-offs (Lambda/Kappa architectures), event-time processing, partitioning strategies, schema evolution (Avro/Protobuf/Parquet), and data correctness at scale.
- Good understanding of data warehousing, dimensional modeling, ETL/ELT design, and distributed computing concepts.
- Excellent verbal and written communication skills.
- Strong interpersonal skills with the ability to collaborate effectively across engineering, product, and customer-facing teams.
Bonus Qualifications
Mobile AdTech Experience
Experience working with Mobile AdTech platforms or digital advertising ecosystems is highly preferred, including knowledge of:
- Direct experience with Real-Time Bidding (RTB), OpenRTB protocol, and Mobile DSP/SSP/Ad Exchange architecture.
- Familiarity with mobile SDK event tracking (impressions, clicks, installs, in-app events) and Mobile Measurement Partner (MMP) integrations.
- Understanding of attribution methodologies (click-through, view-through, multi-touch-based attribution).
- Knowledge of mobile identity solutions (IDFA, GAID, IDFV) and the privacy landscape driving their deprecation.
- Experience with campaign pacing, bid optimization, and budget delivery algorithms.
- Background in ad fraud/IVT detection — click spam, install fraud, SDK spoofing.
Generative AI Experience
Experience building production-grade GenAI applications including:
- LLM integration (OpenAI, Anthropic, Claude, Gemini, Mistral, Cohere, etc.)
- LangChain, LlamaIndex, LangGraph, or similar orchestration frameworks
- Prompt engineering and prompt optimization
- AI evaluation and observability
- AI agents and workflow automation
Additional Preferred Skills
- Experience building scalable, production-grade AI or data platforms.
- Strong SQL optimization and query performance tuning skills.
- Experience running Big Data platforms on-premise — managing Kafka, Spark, and Hadoop/HDFS clusters, capacity planning, and hardware-aware performance tuning in a private data center environment.
- Experience with CI/CD pipelines, Docker, Kubernetes, and containerized application deployment on bare-metal / in-house data center infrastructure.
- Experience with monitoring and observability tools such as Grafana, Prometheus, Datadog, or similar.
- Strong debugging and troubleshooting skills in distributed environments.
- Demonstrated ability to achieve stretch goals in a fast-paced, highly collaborative engineering environment.
- Ability to learn new technologies quickly and independently.
Why Join Us?
- Work on high-scale data infrastructure processing billions of mobile advertising events every day, directly impacting publisher revenue, DSP spend, and advertiser outcomes across the in-app advertising ecosystem.
- Help shape the future of real-time analytics, attribution, campaign pacing, fraud/IVT detection, and GenAI-powered automation for mobile app monetization.
- Build modern data platforms with Spark, Kafka, Hadoop, Go, and Rust on PubMatic's in-house data centers at internet scale, with low latency and high availability.
- Collaborate with industry-leading engineers, product managers, data scientists, ML engineers, DSP partners, and publisher teams.
- Competitive compensation, performance-based incentives, and strong career growth opportunities in a globally recognized AdTech company.
AI-Enabled Engineering Mindset
We value engineers who actively leverage Generative AI tools and IDEs (e.g., GitHub Copilot, ChatGPT, Claude, Cursor, Windsurf) to accelerate development, improve code and data quality, automate repetitive tasks, and enhance documentation and debugging workflows. Engineers who demonstrate AI-first thinking — using these tools to drive faster experimentation, ideation, and technical execution — will thrive in our high-scale, data-intensive environment.
Qualification:
Should have a bachelor's degree in engineering or an equivalent degree from a well-known institute/university.
Additional Information:
Return to Office: PubMatic employees throughout the globe have returned to our offices via a hybrid work schedule (3 days "in office" and 2 days "working remotely") that is intended to maximize collaboration, innovation, and productivity among teams and across functions.
Benefits: Our benefits package includes the best of what leading organizations provide, such as paternity/maternity leave, healthcare insurance, broadband reimbursement. As well, when we're back in the office, we all benefit from a kitchen loaded with healthy snacks and drinks and catered lunches and much more!.
Diversity and Inclusion: PubMatic is proud to be an equal opportunity employer; we don't just value diversity, we promote and celebrate it. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status
About PubMatic
PubMatic is one of the world's leading scaled digital advertising platforms, offering more transparent advertising solutions to publishers, media buyers, commerce companies and data owners, allowing them to harness the power and potential of the open internet to drive better business outcomes.
Founded in 2006 with the vision that data-driven decisioning would be the future of digital advertising, we enable content creators to run a more profitable advertising business, which in turn allows them to invest back into the multi-screen and multi-format content that consumers demand.
See All 59 Big Data Engineer Jobs
Find roles that match your experience and apply in just a few clicks.
Find Big Data Engineer JobsBig Data Engineer Job Market
Who's Hiring
Top Industries Hiring
- Technology & Software12
- Consulting & Professional Services2
- Education1
What Employers Look For
The qualifications that appear most often in big data engineer jobs.
- Proficiency in Apache Spark and distributed data processing frameworks
- Experience building and maintaining ETL or ELT pipelines at scale
- Hands-on knowledge of cloud platforms such as AWS, GCP, or Azure
- Strong programming skills in Python, Scala, or Java
- Familiarity with data warehousing tools such as Snowflake, Redshift, or BigQuery
- Bachelor's degree in computer science, engineering, or a related technical field
Tips for Your Big Data Engineer Job Search
Tailor your resume to stack keywords
Big data engineer postings vary widely by stack. Scan each job description for the specific tools listed, Spark, Kafka, Flink, Databricks, and mirror that language in your resume's skills and project sections. Generic lists get filtered out by ATS before a human reads them.
Quantify pipeline scale and throughput
Hiring managers want to know the volume your systems handled. Replace vague phrases like 'built data pipelines' with specifics: terabytes processed daily, ingestion latency targets hit, or cost reductions from optimization work. Numbers make your impact concrete and comparable.
Apply early to roles that fit
Migrate Mate lists big data engineer openings from across the United States in one place, so you can find roles that match and apply directly to each listing.
Target postings by cloud platform specialization
Many teams are platform-first: they want AWS EMR, Google Dataproc, or Azure HDInsight depth, not broad familiarity. When you filter openings, prioritize the platform you know deepest. Generalist applications to platform-specific roles rarely advance past the first screen.
Prepare a system design answer for interviews
Almost every big data engineer interview includes a design round where you architect a pipeline end to end. Practice narrating trade-offs out loud: batch vs. streaming, storage format choices, partitioning strategy. Interviewers score your reasoning process, not just the final design.
Negotiate compensation using total package framing
Big data engineer offers often include equity, signing bonuses, and cloud certification reimbursements alongside base pay. When negotiating, ask for the full breakdown before countering. A lower base at a company with strong equity and training budget can outperform a higher flat offer.
Big Data Engineer Jobs: Frequently Asked Questions
Which companies are hiring the most big data engineers?
The most active employers for big data engineers right now are TikTok, Esolvit, and JPMorganChase, and the most openings are in Virginia, California, and Texas, based on current listings on Migrate Mate as of November 2026. Financial services, healthcare technology, and large-scale retail platforms tend to post the highest volumes consistently.
How many big data engineer jobs are remote?
About 47% of big data engineer openings are fully remote or hybrid as of November 2026, reflecting the infrastructure-heavy nature of the role. Pipeline development, cloud architecture, and data modeling work are most commonly offered remotely, while roles involving on-premises Hadoop clusters or regulated data environments more often require on-site presence.
How do you become a big data engineer?
Start by building proficiency in a core programming language like Python or Scala, then learn a distributed processing framework such as Apache Spark. Practice designing and deploying pipelines on a free-tier cloud environment to build a portfolio. Earning a cloud data engineering certification strengthens your candidacy, and contributing to open-source data tooling projects demonstrates real-world depth.
Can you get hired as a big data engineer without prior experience?
Entry-level big data engineer roles do exist, and candidates break in by demonstrating hands-on project work rather than years of tenure. Build a public portfolio with a documented end-to-end pipeline project using open datasets. Roles titled data engineer or junior data platform engineer are common starting points before moving into big data specialization.
What does the big data engineer interview process look like?
Most processes run three to five rounds. An initial screen with a recruiter or hiring manager is followed by a technical phone or video round covering Python or Scala coding. A system design round asks you to architect a data pipeline from ingestion to serving. Some teams add a take-home exercise or a final loop with cross-functional stakeholders before extending an offer.
Where can I find and apply to big data engineer jobs?
You can find and apply to big data engineer jobs on Migrate Mate, which lists current openings from across the United States. Search the listings to find roles that match your stack and experience level, then apply directly to each one that fits.
See All 59 Big Data Engineer Jobs
Find roles that match your experience and apply in just a few clicks.
Find Big Data Engineer Jobs