AI Architect

AI Feature Blueprinting: Create technical blueprints for AI features, including architecture, data needs, model strategy, evaluation approach, and delivery risks.

Model Selection: Define patterns for identifying which LLM model should be used for each call, and when to use NLP/NLU instead.

Harness Architecture: Shape reusable patterns for agent loops, tool/function calling, context construction, memory, structured outputs, retries, and failure handling.

Prompt Engineering: Set standards for prompt design, versioning, review, experimentation, and regression testing as first-class engineering assets.

Evaluation: Design eval frameworks, golden datasets, regression suites, error analysis, and LLM-as-judge patterns to measure quality and guide decisions.

RAG & Knowledge Patterns: Guide document parsing, chunking, embeddings, retrieval, reranking, prompt assembly, and grounded response patterns.

Mentorship & Enablement: Coach P2/P3 AI engineers, review technical designs, create examples, and help junior engineers present architecture findings clearly.

Cross-Guidewire Collaboration: Partner with Product, AI CoE, PS, customer teams, and SI partners.

KPIs & Success Metrics

AI feature blueprints are clear, reusable, and actionable for Strike Teams.

Model and architecture choices are backed by evals, analysis, and documented trade-offs.

Prompt, harness, RAG, and evaluation patterns are adopted consistently by P2/P3 engineers.

Delivered AI features meet project bars for accuracy, groundedness, reliability, latency, and cost.

Junior and mid-level engineers can implement patterns and explain the results confidently.

Key Skills & Experience

Education: Bachelor's or Master's degree in Computer Science, Data Science, Statistics, Machine Learning, Engineering, or a related technical field.

Experience: 8+ years of experience in AI, machine learning, data science, software engineering, or solution architecture.

AI Architecture: Strong ability to translate ambiguous product or delivery goals into practical AI system designs.

AI Fundamentals: Deep understanding of LLMs, prompts, embeddings, retrieval, structured outputs, tool use, agentic workflows, and model failure modes.

NLP/NLU Judgment: Able to recognize when classical NLP/NLU, rules-based approaches, lightweight ML, or structured extraction are better than generative AI.

Technical Proficiency:

Strong Python/Java skills and ability to read, review, and contribute to production-quality code.

Experience designing LLM harnesses, including orchestration, tool/function calling, context construction, retries, and fallback behavior.

Strong prompt engineering practice, including structured iteration, versioning, evaluation, and regression testing.

Working knowledge of RAG patterns, embeddings, retrieval evaluation, and vector search concepts.

Comfortable designing evaluation approaches using golden sets, regression suites, error taxonomies, and human review loops.

Engineering Excellence: Familiar with Git, code review, testing, CI/CD concepts, cloud-based delivery, and production engineering practices.

Communication & Mentorship: Able to explain complex AI trade-offs clearly and coach less experienced engineers without direct reporting authority.

Preferred Skills & Experience

Agentic AI Systems: Experience with agentic frameworks or custom agent harnesses in production.

Evals: Experience building structured eval harnesses, regression suites, LLM-as-judge patterns, or production feedback loops.

Cloud: Hands-on AWS experience with services such as Bedrock, SageMaker, OpenSearch, or related cloud-native AI/data services.

Fine-Tuning: Experience evaluating or implementing model fine-tuning, distillation, or domain adaptation.

Guidewire Knowledge: Familiarity with Guidewire products or the insurance domain is a plus.

Enterprise AI Delivery: Prior experience working on document-heavy, regulated, insurance, finance, or professional services datasets.