Senior Quality Engineer (Forward Deployed)

About the role

Villvay, a boutique agency helping B2B eCommerce companies transform digitally, is looking for a Senior Quality Engineer (Forward Deployed). The role sits at the intersection of hands-on quality engineering, AI and search evaluation, and client collaboration. You will work alongside clients and delivery teams on live systems such as search, product and AI platforms, delivering rigorous deterministic testing across web, mobile and API, and non-deterministic testing by defining quality, evals and presenting evidence, risks and mitigation plans.

Key responsibilities

• Design, run and analyse evals using agreed benchmarks, samples and baselines, and define measurement criteria for non-deterministic cases and agentic components
• Present evaluation results to clients and stakeholders, highlighting risks and mitigation plans
• Maintain locked baselines and approved sources of truth, and report reproducible defects with evidence
• Run functional, regression and exploratory testing of web and mobile apps across devices and platforms, and write test cases with agentic AI tools
• Build a shift-left quality engineering discipline with Engineering, Product and DevOps
• Design and run frontend and backend performance tests and identify bottlenecks
• Represent Villvay to clients and stakeholders

Required profile

• 4 to 6 years of experience in software QA or test engineering
• Experience with quality measurement, evaluation and automation frameworks
• Ability to present to clients and stakeholders, with clear written and verbal communication
• Strong multitasking and context-switching across concurrent workstreams
• Practical use of AI-assisted tools (Cursor, Claude, Rovo or equivalent) with structured prompts

Required skills

• Deterministic and non-deterministic testing, web, mobile and API functional testing (automation and manual)
• E-commerce search evaluation experience is an added advantage
• Hands-on experience with Elasticsearch, e-commerce platforms, automation and performance testing tools is an added advantage

What we offer

• Competitive salaries for those who meet the high bar
• Hybrid work model
• Paid parental leave and paid menstrual leave
• Flat structure, OPD and insurance coverage