Engineer
Qualcomm
Bengaluru, Karnataka, India
Posted on May 21, 2025
Company
Qualcomm India Private Limited
Job Area
Engineering Group, Engineering Group > Systems Test Engineering
General Summary
Job Description
We are seeking a Engineer AI System-Level Test Engineer to lead end-to-end testing of Retrieval-Augmented Generation (RAG) AI systems for Hybrid, Edge-AI Inference solutions. This role will focus on designing, developing, and executing comprehensive test strategies for evaluating the reliability, accuracy, usability and scalability of large-scale AI models integrated with external knowledge retrieval systems.
The ideal candidate needs to have deep expertise in AI testing methodologies, experience with large language models (LLMs), expertise in building test solutions for AI Inference stacks, RAG, search/retrieval architecture, and a strong background in automation frameworks, performance validation, and building E2E automation architecture.
Experience testing large-scale generative AI applications, familiarity with LangChain, LlamaIndex, or other RAG-specific frameworks, and knowledge of adversarial testing techniques for AI robustness are preferred qualifications
Key Responsibilities
Test Strategy & Planning
Minimum Qualifications
Qualcomm expects its employees to abide by all applicable policies and procedures, including but not limited to security and other requirements regarding protection of Company confidential information and other confidential and/or proprietary information, to the extent those requirements are permissible under applicable law.
To all Staffing and Recruiting Agencies: Our Careers Site is only for individuals seeking a job at Qualcomm. Staffing and recruiting agencies and individuals being represented by an agency are not authorized to use this site or to submit profiles, applications or resumes, and any such submissions will be considered unsolicited. Qualcomm does not accept unsolicited resumes or applications from agencies. Please do not forward resumes to our jobs alias, Qualcomm employees or any other company location. Qualcomm is not responsible for any fees related to unsolicited resumes/applications.
If you would like more information about this role, please contact Qualcomm Careers.
3074793
Qualcomm India Private Limited
Job Area
Engineering Group, Engineering Group > Systems Test Engineering
General Summary
Job Description
We are seeking a Engineer AI System-Level Test Engineer to lead end-to-end testing of Retrieval-Augmented Generation (RAG) AI systems for Hybrid, Edge-AI Inference solutions. This role will focus on designing, developing, and executing comprehensive test strategies for evaluating the reliability, accuracy, usability and scalability of large-scale AI models integrated with external knowledge retrieval systems.
The ideal candidate needs to have deep expertise in AI testing methodologies, experience with large language models (LLMs), expertise in building test solutions for AI Inference stacks, RAG, search/retrieval architecture, and a strong background in automation frameworks, performance validation, and building E2E automation architecture.
Experience testing large-scale generative AI applications, familiarity with LangChain, LlamaIndex, or other RAG-specific frameworks, and knowledge of adversarial testing techniques for AI robustness are preferred qualifications
Key Responsibilities
Test Strategy & Planning
- Define end-to-end test strategies for RAG, retrieval, generation, response coherence, and knowledge correctness
- Develop test plans & automation frameworks to validate system performance across real-world scenarios.
- Hands-on experience in benchmarking and optimizing Deep Learning Models on AI Accelerators/GPUs
- Implement E2E solutions to integrate Inference systems with customer software workflows
- Identify and implement metrics to measure retrieval accuracy, LLM response quality
- Build automated pipelines for regression, integration, and adversarial testing of RAG workflows.
- Validate search relevance, document ranking, and context injection into LLMs using rigorous test cases.
- Collaborate with ML engineers and data scientists to debug model failures and identify areas for improvement.
- Conduct scalability and latency tests for retrieval-heavy applications. Analyze failure patterns, drift detection, and robustness against hallucinations and misinformation.
- Work closely with AI research, engineering teams & customer teams to align testing with business requirements.
- Generate test reports, dashboards, and insights to drive model improvements.
- Stay up to date with the latest AI testing frameworks, LLM evaluation benchmarks, and retrieval models.
- 2+ years of experience in AI/ML system testing, software quality engineering, or related fields.
- Bachelor’s or master’s degree in computer science engineering/ data science / AI/ML
- Hands-on experience with test automation frameworks (e.g., PyTest, Robot Framework, JMeter).
- Proficiency in Python, SQL, API testing, vector databases (e.g., FAISS, Weaviate, Pinecone) and retrieval pipelines.
- Experience with ML model validation metrics (e.g., BLEU, ROUGE, MRR, NDCG).
- Expertise in CI/CD pipelines, cloud platforms (AWS/GCP/Azure), and containerization (Docker, Kubernetes).
- Work on cutting-edge AI retrieval-augmented generation technologies
- Collaborate with world-class AI researchers and engineers.
Minimum Qualifications
- Bachelor's degree in Engineering, Information Systems, Computer Science, or related field.
Qualcomm expects its employees to abide by all applicable policies and procedures, including but not limited to security and other requirements regarding protection of Company confidential information and other confidential and/or proprietary information, to the extent those requirements are permissible under applicable law.
To all Staffing and Recruiting Agencies: Our Careers Site is only for individuals seeking a job at Qualcomm. Staffing and recruiting agencies and individuals being represented by an agency are not authorized to use this site or to submit profiles, applications or resumes, and any such submissions will be considered unsolicited. Qualcomm does not accept unsolicited resumes or applications from agencies. Please do not forward resumes to our jobs alias, Qualcomm employees or any other company location. Qualcomm is not responsible for any fees related to unsolicited resumes/applications.
If you would like more information about this role, please contact Qualcomm Careers.
3074793