Senior Data Engineer

Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States. This is a fantastic opportunity to join an established and well-respected organization offering tremendous career growth potential. **Job Title:** Senior Data Engineer **Location:** 100% Remote (U.S.) **Position Type:** Full-time, Direct W2 **Salary Range:** $100,000–$150,000 Annually **Experience Required:** 6+ years **Sponsorship:** U.S. Citizens, Green Card Holders, EAD Holders, and H-1B transfer candidates are encouraged to apply. We are unable to sponsor new H-1B visa petitions for this position. **Job Summary:** We are seeking an accomplished **Senior Data Engineer** to architect, design, develop, and maintain enterprise-grade data platforms, scalable data pipelines, and distributed data processing systems that support analytics, business intelligence, and machine learning initiatives across multiple business domains. In this role, you will be responsible for the end-to-end data engineering lifecycle, from translating business and analytical requirements into robust data architectures, to developing reliable ETL/ELT pipelines, to deploying cloud-native data solutions and supporting them throughout their operational lifespan. The successful candidate will bring deep expertise in data engineering, distributed computing, cloud data platforms, and database technologies, combined with strong hands-on experience building scalable, secure, and high-performance data solutions. You will work closely with data scientists, business analysts, software engineers, solution architects, DevOps engineers, and stakeholders in an Agile environment to deliver high-quality, reliable, and governed data platforms that directly support strategic business outcomes. ***Key Responsibilities*** - Design, build, and continuously refine scalable batch and real-time data pipelines using Python, SQL, Spark, Scala, or equivalent technologies, ensuring reliable, efficient, and high-performance data movement across enterprise systems while supporting evolving business and analytical requirements. - Author secure, reusable, and production-quality ETL/ELT workflows that adhere to enterprise coding standards, data governance policies, data quality principles, and security best practices, incorporating validation, encryption, auditing, and error handling throughout the data lifecycle. - Develop scalable data integration solutions using modern cloud data platforms such as AWS, Azure, or Google Cloud, leveraging services including Databricks, Snowflake, BigQuery, Redshift, Synapse Analytics, Data Factory, Glue, or equivalent technologies to enable enterprise data processing. - Design and implement robust data architectures, dimensional data models, data lakes, data warehouses, and streaming data solutions that integrate multiple structured, semi-structured, and unstructured data sources while ensuring consistency, scalability, and high availability. - Actively participate in enterprise data architecture discussions, cloud migration initiatives, technical design reviews, and solution planning sessions by evaluating trade-offs involving scalability, performance, maintainability, governance, security, and operational costs. - Continuously monitor, profile, and optimize ETL processes, Spark jobs, SQL queries, database performance, storage utilization, partitioning strategies, and pipeline throughput by identifying bottlenecks and implementing measurable performance improvements. - Implement and maintain robust metadata management, data cataloging, lineage tracking, schema evolution, data quality validation, monitoring, and governance frameworks that ensure trusted, discoverable, and compliant enterprise data assets. - Develop comprehensive automated testing frameworks for data pipelines, ETL workflows, data validation, reconciliation, integration testing, and performance testing using modern testing m…