Engenheiro de Dados Sênior| IA Generativa, Dados e MDM

**This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Engenheiro de Dados Sênior | IA Generativa, Dados e MDM based in Brazil.** This role offers the opportunity to design and evolve advanced data solutions powered by Artificial Intelligence, Generative AI, and modern data engineering practices. You will work on strategic initiatives involving customer data platforms, intelligent agents, and large-scale data processing environments. The position combines data engineering, machine learning, governance, and automation to create solutions that generate measurable business impact. You will contribute to building scalable architectures, improving data quality, and enabling smarter decision-making through trusted information. The environment encourages collaboration between technology, data, and business teams while promoting innovation and continuous improvement. You will have the opportunity to work with cutting-edge technologies such as Databricks, Spark, LLMs, and intelligent workflows. ### Accountabilities: As a Senior Data Engineer, you will be responsible for designing, developing, and optimizing scalable data solutions that support AI-driven initiatives, data governance, and business intelligence. Your responsibilities will include: - Developing, optimizing, and maintaining scalable data pipelines using Databricks, Spark, and PySpark; - Integrating, transforming, processing, and making customer data available across large-scale environments; - Building and maintaining cloud-based Big Data products, ensuring scalability, performance, reliability, and data quality; - Supporting initiatives related to customer data management, Golden Record, data quality, governance, and MDM; - Designing data solutions that enable Generative AI applications, autonomous agents, multi-agent systems, and intelligent workflows; - Preparing and structuring data for consumption by Machine Learning models, LLMs, AI agents, and automated processes; - Supporting the development of intelligent agents capable of analyzing large volumes of customer data; - Identifying patterns, inconsistencies, duplicates, anomalies, and opportunities for improvement within data assets; - Developing mechanisms for alerts, explainable recommendations, and data-driven decision support; - Applying Analytics and Machine Learning techniques for anomaly detection, classification, clustering, and scoring; - Supporting data enrichment, prioritization, and governance strategies; - Exploring and implementing architectures such as RAG, autonomous agents, and intelligent workflows; - Working with near real-time data processing solutions using Kafka; - Supporting Machine Learning model production in Databricks through MLOps practices; - Creating and maintaining CI/CD pipelines using GitHub and GitHub Actions; - Ensuring engineering best practices related to quality, efficiency, maintainability, testing, and governance; - Collaborating with Data, Technology, MDM, and Business teams to deliver measurable and applicable solutions; - Participating in technical refinements, process analysis, and AS-IS/TO-BE documentation; - Identifying technical debt and proposing continuous improvements in architecture, processes, and data solutions. ## Requirements: We are looking for a senior professional with strong experience in data engineering, cloud environments, and AI-driven data solutions. The ideal candidate should have: - Solid experience as a Data Engineer, Senior Data Engineer, or similar role; - Strong knowledge of Python for data engineering, automation, analytics, and AI solution development; - Advanced SQL skills and experience working with structured data exploration; - Hands-on experience with Databricks, PySpark, and Apache Spark; - Proven experience building, optimizing, and maintaining scalable data pipelines; - Experience working with cloud environments, preferably Azure Databricks; - Knowledge or experie…