Supervisor, Test Engineering

## Responsibilities **Position Overview:** The Test Engineering team is the quality assurance backbone of the CMS Medicare Systems Solutions (CMSS) portfolio, responsible for protecting the stability and integrity of FISS, MCS, DME, CWF, and Crosscutting programs through every software release. These systems process hundreds of millions of claims annually across 15+ million lines of legacy COBOL. The Supervisor, Test Engineering owns the end-to-end test strategy for this environment, with a primary mandate to ensure that every change, enhancement, and modernization effort is validated against the full depth of legacy system behavior before it reaches production. In this role, the Supervisor applies an expert understanding of how the test engineering function integrates with development, architecture, operations, and business teams to protect system stability and support CMS's modernization objectives. The Supervisor identifies and solves technical and operational testing problems, particularly those arising at the intersection of legacy mainframe systems and modern integration layers. The Supervisor develops, coaches and directs test engineers and functional testers to adopt consistent regression and automation standards, adapting test plans and priorities to address resource and operational challenges, leading the test engineering team, and focusing on achieving quality, test coverage, and release readiness objectives. **Key Responsibilities:** **Mainframe & Regression Testing Leadership** - Develop and implement a comprehensive testing strategy grounded in Agile/DevOps practices and automation-first principles. - Own and continuously mature the regression testing strategy for COBOL-based mainframe systems, ensuring comprehensive coverage of all claims processing logic, edits, and business rules across every release cycle. - Maintain and expand the regression test suite to reflect changes introduced by enhancements, new payment models, requirements updates, and modernization activities. - Establish and enforce regression baseline standards: define what "passing" looks like for each system, document known behaviors, and ensure deviations are flagged, triaged, and resolved before production deployment. - Lead mainframe-specific test planning for batch processing, JCL job streams, VSAM file handling, and claims adjudication logic. - Coordinate regression test execution across all 12 Medicare Administrative Contractors (MACs) to ensure consistent validation of system behavior in distributed environments. - Ensure regression test assets are version-controlled, traceable to requirements and business rules, and maintained as living documentation of system behavior. **Test Planning & Execution** - Create, maintain, and manage test execution plans spanning multiple testing phases, environments, and release cycles. - Coordinate and oversee automated and manual testing activities across integrated teams and subcontractors. - Ensure test environments, data sets, and automation assets support scalable, repeatable testing. - Participate in requirements and user story reviews and design walk-throughs to ensure testability. **Systems Stability & Release Readiness** - Serve as the quality gate for every production release, including Go/No-Go recommendations grounded in complete, documented test evidence. - Develop and maintain release-specific test plans that account for the full scope of changes, including interactions between legacy components and any modernized or integrated services. - Identify and escalate systemic risks to system stability, including gaps in test coverage, untested code paths, and regression exposure introduced by modernization work. - Lead defect triage and resolution analysis, ensuring defects are prioritized by business impact and that production readiness decisions are made with full visibility into open risk. - Maintain production readiness checklists, test completion criteria, and sign-off documentation for each…