Principal Software Engineer (Data Engineering)

Job Description

We are seeking a highly skilled **Principal Software Engineer – Data Engineering** to design, develop, and optimize scalable data platforms and pipelines. The ideal candidate will have strong expertise in **Python, SQL, Snowflake (or equivalent cloud data warehouse), cloud technologies, CI/CD practices, and AI-enabled data solutions**. This role requires technical leadership, architecture ownership, mentoring capabilities, and hands-on engineering excellence.

Key Responsibilities

Data Engineering & Platform Development

Design and implement scalable, reliable, and secure data pipelinesDevelop ETL/ELT solutions for processing large-scale structured and unstructured datasetsBuild data ingestion frameworks supporting batch and near real-time processingDesign and optimize data models for analytics, reporting, and machine learning workloadsEnsure high data quality, governance, observability, and reliability across data platformsCloud & Data Warehouse

Architect and maintain cloud-native data solutions on AWS or Azure or GCPDevelop and optimize solutions using **Snowflake** or equivalent cloud data warehouse technologiesImplement data lake and lakehouse architecturesOptimize storage costs, compute utilization, and query performanceSoftware Engineering Excellence

Develop production-grade applications and frameworks using PythonPerform code reviews and establish engineering best practicesDrive architectural decisions and technology selectionEnsure adherence to security, scalability, and reliability standardsCI/CD & DevOps

Design and implement CI/CD pipelines for data and application deploymentsAutomate build, test, deployment, and monitoring processesManage Infrastructure as Code (IaC) using Terraform, CloudFormation, or similar toolsImprove deployment reliability and observabilityAI & Advanced Analytics

Enable AI/ML workflows by building robust feature and training data pipelinesWork with AI teams to operationalize machine learning modelsExplore Generative AI, LLM integrations, vector databases, and AI-driven automation solutionsSupport model monitoring, governance, and deployment pipelinesLeadership & Mentoring

Provide technical leadership across multiple engineering teamsMentor senior engineers and establish engineering standardsCollaborate with architects, product managers, and business stakeholdersDrive innovation and strategic technology initiatives

Qualifications

10+ years of experience in Data Engineering.

Strong expertise in **Python programming**Advanced SQL development and performance tuning skillsExperience with **Snowflake** or equivalent cloud data warehouse technologiesStrong understanding of data modeling, ETL/ELT frameworks, and data governanceExperience with cloud platforms: Microsoft Azure (Preferred) AWS Google Cloud PlatformDevOps & CI/CD

Experience with: Azure DevOps, GitHub Actions, Jenkins, GitLab CI/CD, Knowledge of Docker and Kubernetes., Infrastructure as Code experience

AI & Modern Data Technologies

Exposure to:

Machine Learning pipelines Generative AI concepts LLM integrations Vector databases RAG architectures Prompt engineering basics

Preferred Qualifications:

Experience building enterprise-scale data platformsKnowledge of Spark, Databricks, Kafka, or streaming technologiesUnderstanding of Data Mesh and modern data architecture principlesExperience in observability and monitoring toolsSnowflake, Azure, AWS, or GCP certifications are a plus

Desired Competencies

Technical leadershipSolution architectureStrategic thinkingStakeholder managementProblem-solvingMentoring and coachingStrong communication skillsAgile development practices

Success Metrics

Reliable and scalable data platform deliveryReduced pipeline failures and improved SLA complianceImproved data quality and governanceFaster deployment cycles through CI/CD automationSuccessful adoption of AI-enabled data solutionsIncreased engineering productivity and platform efficiency

Additional Information

Our Benefits

Flexible working environmentVolunteer time offLinkedIn LearningEmployee-Assistance-Program (EAP)NIQ may utilize artificial intelligence (AI) tools at various stages of the recruitment process, including résumé screening, candidate assessments, interview scheduling, job matching, communication support, and certain administrative tasks that help streamline workflows. These tools are intended to improve efficiency and support fair and consistent evaluation based on job-related criteria. All use of AI is governed by NIQ’s principles of fairness, transparency, human oversight, and inclusion. Final hiring decisions are made exclusively by humans. NIQ regularly reviews its AI tools to help mitigate bias and ensure compliance with applicable laws and regulations. If you have questions, require accommodations, or wish to request human review were permitted by law, please contact your local HR representative. For more information, please visit NIQ’s AI Safety Policies and Guiding Principles: https://nielseniq.com/global/en/info/niqs-ai-safety-policies/

About NIQ

NIQ is the world’s leading consumer intelligence company, delivering the most complete understanding of consumer buying behavior and revealing new pathways to growth. In 2023, NIQ combined with GfK, bringing together the two industry leaders with unparalleled global reach. With a holistic retail read and the most comprehensive consumer insights—delivered with advanced analytics through state-of-the-art platforms—NIQ delivers the Full View™. NIQ is an Advent International portfolio company with operations in 100+ markets, covering more than 90% of the world’s population.

For more information, visit NIQ.com

Want to keep up with our latest updates?

Follow us on: LinkedIn | Instagram | Twitter | Facebook

Our commitment to Diversity, Equity, and Inclusion

At NIQ, we are steadfast in our commitment to fostering an inclusive workplace that mirrors the rich diversity of the communities and markets we serve. We believe that embracing a wide range of perspectives drives innovation and excellence. All employment decisions at NIQ are made without regard to race, color, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, age, disability, genetic information, marital status, veteran status, or any other characteristic protected by applicable laws. We invite individuals who share our dedication to inclusivity and equity to join us in making a meaningful impact. To learn more about our ongoing efforts in diversity and inclusion, please visit the https://nielseniq.com/global/en/news-center/diversity-inclusion