Senior Test Specialist

Job Summary

Job Description: Senior AI Application Tester (10+ Years Experience)
Role Overview
We are seeking a highly experienced AI Application Tester to lead quality assurance for AI-driven applications. The role focuses on functional validation, performance (load & stress) testing, and AI model/system reliability to ensure enterprise-grade quality, scalability, and compliance.
This position requires deep expertise in AI/ML system testing, non-deterministic validation, and high-scale performance engineering.
________________________________________

Key Responsibilities

1. Functional Testing (AI Systems)

• Design and execute end-to-end functional test strategies for AI applications (LLMs, ML models, intelligent workflows)
• Validate business logic, user workflows, and system behavior against requirements [aqua-cloud.io]
• Develop test cases for deterministic + non-deterministic outputs (AI responses, predictions)
• Perform integration and system testing across APIs, data pipelines, and UI layers [jobed.ai]

2. Performance, Load & Stress Testing

• Design and execute performance test strategies (load, stress, endurance)
• Validate application performance under peak and extreme conditions to identify bottlenecks [mckinsey.com]
• Simulate high concurrency, large data volumes, and model inference loads
• Analyze:
• Response time
• Throughput
• Resource utilization (CPU, memory, GPU)
• Identify system breaking points and scalability limits

3. AI/ML Model Validation

• Validate model accuracy, consistency, and reliability across datasets [linkedin.com]
• Perform:
• Edge-case testing
• Bias/fairness validation
• Explainability testing
• Conduct scenario-based and adversarial testing (prompt manipulation, unexpected inputs)
• Ensure AI outputs meet quality, safety, and compliance standards

4. Test Automation & Framework Development

• Build and maintain automated test frameworks for functional and performance testing
• Implement AI-assisted testing tools for test generation and execution [geeksforgeeks.org]
• Integrate testing into CI/CD pipelines for continuous validation
• Develop reusable test data strategies and simulation environments

5. Defect Management & Quality Governance

• Identify, document, and track defects, anomalies, and performance issues [interviewguy.com]
• Perform root cause analysis and recommend fixes
• Ensure adherence to enterprise quality standards, compliance, and security requirements
• Provide test reports, dashboards, and quality metrics

6. Collaboration & Leadership

• Work closely with:
• AI/ML engineers
• Data scientists
• Product and platform teams [torchora.com]
• Define testing strategy and QA best practices
• Mentor junior QA/test engineers and lead testing initiatives
• Drive adoption of AI testing frameworks and modern QA practices

Skill Requirements

Experience

• 10+ years in Software QA / Testing
• 3–5+ years in AI/ML system testing
• Experience in large-scale distributed systems / cloud applications

Technical Skills

• Strong expertise in:
• Functional testing (manual + automation)
• Performance testing (Load, Stress, Endurance)
• Hands-on with tools:
• Selenium, JMeter, LoadRunner
• API testing (Postman, REST Assured)
• Programming:
• Python
• CI/CD integration:
• Azure DevOps

AI/ML Testing Skills (Nice to have)

• Understanding of:
• ML lifecycle & model evaluation metrics
• Data validation and pipeline testing
• Experience with:
• Model validation, fairness testing, and output evaluation
• Ability to test probabilistic systems vs deterministic software

Performance Engineering Skills

• Expertise in:
• Load modeling & workload simulation
• Bottleneck identification and tuning
• Experience with scalable cloud architectures (Azure/AWS)

Soft Skills

• Strong analytical and problem-solving skills
• Ability to design complex test scenarios and edge cases
• Excellent stakeholder communication
• Leadership and mentoring capability

Nice to Have

• Experience with:
• LLM testing / GenAI systems
• Observability tools (Prometheus, Grafana, App Insights)
• AI testing frameworks (Langfuse, custom evaluation pipelines)

Summary (Slide-ready)

• Role: Senior AI QA / Application Tester
• Focus: Functional + Stress/Performance Testing
• Scope: AI/ML systems, APIs, distributed applications
• Experience: 10+ years QA + AI testing exposure
• Outcome: Reliable, scalable, and compliant AI applications

Other Requirements