Principal Software Engineer - Model Inferencing
Microsoft
| Help build the end-to-end Generative AI stack for Responsible AI at Microsoft. Lead the development of global, enterprise-scale systems for LLM hosting, inference orchestration, and agentic workflows to ensure AI systems are safe, reliable, and grounded. We are looking for experts in distributed systems and high-performance backend engineering to solve complex challenges in AI alignment, task adherence, and autonomous tool-use at scale.
Microsoft is a company where passionate innovators come together to collaborate, envision new possibilities, and advance their careers. In an AI-first world, we focus on driving meaningful innovation through openness and teamwork.
The CoreAI organization builds the fundamental AI backbone for Microsoft’s flagship products, including GitHub, Office, Teams, and Windows. Within this group, the Responsible AI team is at the forefront of Azure AI innovation, building the global infrastructure required to run the world's largest AI workloads safely. Our mission is to identify, measure, and mitigate risks across all content modalities.
As a Principal Software Engineer, you will join a talented multidisciplinary team of engineers, scientists, and product managers to build industry-leading AI services. You will architect the infrastructure required for agent workflow management and real-time safety guardrails. This role is central to hosting and orchestrating models at scale, integrating deeply with platforms such as Microsoft Foundry, Azure AI Content Safety, and Azure OpenAI Service. |
Responsibilities
- Own and lead the architecture for complex, high-availability AI services, ensuring scalability, resiliency, and low-latency performance.
- Improve AI tools and practices across the SDLC, incorporating Responsible AI controls into the system backbone.
- Lead the integration of new AI services with existing platforms such as Microsoft Foundry, Azure AI Content Safety, and Azure OpenAI Service.
- Mentor teams in producing extensible, secure, and maintainable code. Identify best practices in GenAI coding patterns and drive high-quality validation strategies.
- Identify and manage upstream/downstream dependencies, collaborating with partner teams to ensure seamless end-to-end testing and live site coverage.
- Act as a lead for security-by-design, ensuring AI safety features are implemented and regulatory audit trails are maintained.
Qualifications
Required Qualifications:
- Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.
Other Requirements:
Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
- Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Preferred Qualifications:
- Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.
- Proven experience in building high-scale distributed systems and high-availability services.
- Deep understanding of the AI lifecycle, specifically regarding model inference and system-level optimization.
Software Engineering IC5 - The typical base pay range for this role across the U.S. is USD $139,900 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.