Member of Technical Staff - QA Engineer

Patronus AI•San Francisco, CA

About The Position

You are responsible for ensuring the quality and reliability of our products and systems. You will work closely with our engineering, AI and design teams to identify and resolve any potential issues or bugs, create testing plans and datasets, ensuring that our products meet the highest standards of performance and functionality. Successful applicants are skilled, have prior QA experience, and are extremely detail-oriented. In this role, you will: Develop and execute test plans to ensure the quality of software features. Create high quality, specific test cases that cover diverse scenarios and edge cases to catch unexpected behavior. Ensure that production releases are well tested. Typically releases happen on a weekly cadence; QA will ensure reliability of new features in the web platform, collaborating closely with frontend engineers Regularly report bugs and issues from testing results and file tasks; Triage to engineers accordingly Collaborate with the engineering and AI teams to understand requirements for testing; Participate in technical design meetings as needed Construct datasets for AI research, working with the AI team, including datasets for model training, fine-tuning and evaluation Develop and maintain automated testing scripts for our API Help customer success team investigate and troubleshoot customer-reported issues to identify root causes and recommend solutions Executing load tests Run benchmarking scripts to assess evaluator performance from configuration changes, e.g. a change in the model selection, underlying dataset or other parameters

Requirements

3+ years of experience as a QA engineer working on web platforms in a production environment
Minimum of at least 2 year of experience working on test automation with Python or Javascript
Experience with REST APIs
Experience performing manual application tests
Experience writing end-to-end tests
Experience with Playwright, Selenium, Cypress
Experience with tools like Postman, Swagger, OpenAPI
Have good character, integrity, and respect for others!

Responsibilities

Develop and execute test plans to ensure the quality of software features.
Create high quality, specific test cases that cover diverse scenarios and edge cases to catch unexpected behavior.
Ensure that production releases are well tested.
Regularly report bugs and issues from testing results and file tasks; Triage to engineers accordingly
Collaborate with the engineering and AI teams to understand requirements for testing; Participate in technical design meetings as needed
Construct datasets for AI research, working with the AI team, including datasets for model training, fine-tuning and evaluation
Develop and maintain automated testing scripts for our API
Help customer success team investigate and troubleshoot customer-reported issues to identify root causes and recommend solutions
Executing load tests
Run benchmarking scripts to assess evaluator performance from configuration changes, e.g. a change in the model selection, underlying dataset or other parameters