Data Engineer - Assistant Manager

Senior Associate, Data Engineer

Data, Analytics & AI Services (DAAIS)

Role Summary

State Street Investment Management's Data, Analytics & AI Services (DAAIS) team is seeking a Senior Associate, Data Engineer for our Bangalore, India office. This role is responsible for developing, enhancing, and supporting modern data platform capabilities that enable the investment management business. The successful candidate will have hands-on experience in data engineering, distributed data processing, cloud-based data platforms, and software development practices.

Working closely with senior engineers, architects, and business stakeholders, the individual will contribute to the design, development, testing, and operational support of scalable data pipelines and data products across the enterprise data ecosystem.

Key Responsibilities

Develop and maintain batch and streaming data pipelines using PySpark, Spark SQL, and Databricks.

Build and support data processing solutions following Bronze/Silver/Gold (multi-hop) architecture patterns.

Implement and maintain data ingestion pipelines from multiple sources including relational databases, APIs, event streams, and cloud object storage.

Develop incremental processing, Change Data Capture (CDC), and data transformation workflows.

Write high-quality, scalable, and maintainable code adhering to engineering standards and best practices.

Participate in code reviews, testing, deployment, and production support activities.

Monitor data pipeline performance and assist in troubleshooting, root cause analysis, and issue resolution.

Support implementation of data quality, metadata, and governance requirements.

Required Qualifications

3-7 years of experience in software engineering, data engineering, or related technical roles.

Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field (or equivalent experience).

Hands-on experience with Databricks, including notebooks, workflows, clusters, and repositories.

Good understanding of Apache Spark concepts including transformations, joins, partitioning, caching, and performance optimization.

Proficiency in Python (PySpark) and SQL.

Experience building and supporting production data pipelines in cloud environments.

Familiarity with software engineering practices including version control, testing, CI/CD, and code management.

Working knowledge of data lake and lakehouse concepts.

Experience with orchestration tools such as Databricks Workflows, Airflow, Azure Data Factory, or similar technologies.

Familiarity with batch, CDC, and streaming ingestion patterns.

Experience working with Azure and/or AWS cloud platforms.

Understanding of cloud storage technologies such as AWS S3 or Azure Data Lake Storage.

Strong analytical, troubleshooting, and problem-solving skills.

Ability to work effectively in a collaborative, global team environment.

Preferred Qualifications

Experience in financial services or asset management environments.

Exposure to Apache Iceberg, Delta Lake, Unity Catalog, or similar governance technologies.

Experience with streaming technologies including Kafka, Event Hubs, Kinesis, or Structured Streaming.

Exposure to data quality, metadata management, and data governance concepts.

Experience supporting machine learning or analytics use cases through data engineering solutions.

Knowledge of monitoring, observability, and operational support practices for data platforms.

About State Street

Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success.

We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work-life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future.

As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.

Discover more information on jobs at StateStreet.com/careers

Read our CEO Statement