AI Quality Engineer

Role Description

In the capacity of an AI Quality Engineer, you are responsible for architecting and managing the intelligent systems that uphold our platform's integrity. Moving beyond manual verification, you will engineer agentic processes and automated solutions to create, sustain, and evaluate test scenarios, while proactively identifying potential issues through production monitoring. By merging LLM capabilities with our core technology—including Python test Suites, Playwright, GitHub Actions, Jira, Sentry, and Grafana—you will develop the essential tooling that scales our quality assurance and engineering efforts. This role works closely with Backend, DevOps, development teams, and product stakeholders, leveraging existing CI/CD, cloud, API, and observability infrastructure to build and own the quality engineering automation layer throughout the software development lifecycle.

Key Responsibilities 

• Architect, implement, and maintain AI-enabled workflows and agents that automate testing, triage, and quality analysis across the software development lifecycle.
• Integrate LLM-based workflows with internal systems through APIs and MCP connectors, including GitHub, Jira, CI/CD pipelines, and observability platforms.
• Engineer AI-driven test lifecycle management systems that extract scenarios from documentation, utilize self-healing mechanisms, and optimize Playwright performance.
• Construct automated anomaly detection pipelines that link performance shifts and user issues across monitoring tools, delivering actionable root-cause intelligence.
• Develop rigorous evaluation frameworks for AI processes, auditing metrics such as precision, latency, and operational expenditure to ensure output reliability.
• Treat prompt design and context engineering as core technical disciplines, ensuring they are version-controlled and thoroughly validated like traditional codebase components.
• Modernize test data orchestration, automating the creation and masking of high-fidelity datasets for non-production environments.
• Drive quality intelligence reporting initiatives, replacing static documentation with interactive dashboards highlighting ROI and automation efficacy.
• Partner with Backend, DevOps, QA, and Product teams to identify and automate high-friction or repetitive activities across the software development lifecycle.
• Deploy AI capabilities while adhering to regulatory and security standards like ISO 27001, maintaining the strict confidentiality of credentials and production assets.
• Ensure dedicated focus on AI architecture and tooling by balancing proactive system development (80%) with targeted support for existing manual QA efforts (20%).
• Develop systems that effectively hand off complex anomalies to the Human QA team, ensuring AI empowers the critical peer-review lifecycle.

Qualifications

• A minimum of three years of professional experience in software, automation, or platform engineering.
• Advanced proficiency in Python for architecting robust tools and services rather than basic scripting.
• Expertise in LLM API implementation, including advanced prompt engineering, RAG, tool calling, and agentic workflows.
• Deep understanding of API orchestration and designing automated systems that interact with live infrastructure.
• Hands-on experience managing CI/CD pipelines, specifically utilizing GitHub Actions.
• Command of SQL for direct data validation and database integrity checks.
• Familiarity with Playwright or similar E2E frameworks, with the ability to optimize suite stability.
• The ability to evaluate non-deterministic AI outputs through rigorous engineering and validation frameworks.
• Knowledge of LLM-specific security vectors, including prompt injection mitigation and automated data sanitization.

Preferred Skills

• Knowledge of JavaScript or TypeScript.
• Experience with AI observability tools such as LangSmith, Promptfoo, or DeepEval.
• Proficiency in monitoring platforms like Grafana and Sentry for proactive alerting.
• Execution of load and performance testing using Locust, k6, or similar utilities.
• Competence in data lifecycle management, including ETL processes and anonymization.
• Advanced usage of Jira and integrated test management software.
• Experience working within highly regulated environments such as ISO 27001 or PCI-DSS.
• Background in mentorship or leadership for quality assurance projects.

Team Collaboration

You will work closely with the IT & Development team, collaborating with QA engineers, designers, developers, data engineers, and other departments such as operations to ensure a seamless customer experience.

Successful candidates will initially participate in our standard QA onboarding shadowing process to deeply understand our existing software development lifecycle before implementing AI-driven optimizations.

Moveo Technologies Corp. (drvn)

As the first and currently the main brand of Moveo Technologies Corporation, drvn provides premium, global, private passenger transportation and logistics services. Providing technology solutions and passenger transportation services for complex ground environments and projects is the company's primary focus.