Principal Data Engineer / Architect - Individual Contributor

**This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Principal Data Engineer / Architect - Individual Contributor based in India.** This role offers the opportunity to design and evolve large-scale data platforms powering modern enterprise applications. You will take ownership of data architecture, engineering strategy, and AI-ready infrastructure for a global technology environment. The position focuses on building scalable, secure, and reliable solutions that transform complex data into actionable insights. You will work across engineering teams to influence product decisions through advanced data, analytics, and AI capabilities. This is a high-impact individual contributor role for a technical leader who enjoys solving complex architecture challenges. You will contribute to the future of data platforms by leveraging cloud technologies, automation, and machine learning operations. The role provides the autonomy, technical depth, and ownership needed to shape next-generation data experiences. ### Accountabilities: The Principal Data Engineer / Architect will lead the design, development, and optimization of enterprise-scale data platforms. This role requires deep technical expertise, strong architectural judgment, and the ability to create reliable data solutions that support analytics, AI, and business innovation. - Design and oversee scalable data architectures, including data platforms, pipelines, storage systems, and processing frameworks. - Develop and maintain robust data models while ensuring data quality, governance, security, and compliance. - Build and optimize high-volume data pipelines for ingesting, transforming, and processing structured and unstructured data. - Architect distributed data systems capable of handling large-scale workloads, including petabyte-level data environments. - Improve platform performance by identifying bottlenecks and implementing solutions for scalability, reliability, and efficiency. - Evaluate emerging data technologies and introduce innovative approaches to strengthen data infrastructure. - Design AI-ready data ecosystems, including semantic layers, metadata structures, context management, and solutions supporting AI agents. - Develop and support AI/ML operations workflows using modern frameworks and platforms. - Collaborate with product engineering teams to influence technical designs around data, analytics, and AI use cases. - Implement data security practices, governance frameworks, and compliance standards across data environments. - Promote engineering excellence through improved development practices, automation, observability, monitoring, and operational reliability. ## Requirements: The ideal candidate is a highly experienced data engineering professional with strong architecture skills and a passion for building modern data platforms. You should have experience solving complex technical challenges in large-scale environments and applying innovative approaches to data, AI, and cloud infrastructure. - Extensive experience designing and building enterprise-grade SaaS solutions with high availability and scalability. - Proven experience handling large-scale structured and unstructured datasets from multiple sources. - Strong background in designing and maintaining data platforms, including distributed computing, storage, orchestration, and streaming infrastructure. - Deep experience with system design involving large-scale data environments, including petabyte-scale architectures. - Hands-on experience with Databricks Lakehouse and modern AI/data infrastructure patterns. - Experience with AI/MLOps workflows, including tools such as MLflow, Databricks AI solutions, Unity Catalog, vector search, and knowledge graph technologies. - Familiarity with AI/ML frameworks such as LangChain and LangGraph for AI workflow integration. - Strong programming skills in Python and SQL, with experience in Java or Scala…