📍 Pune, IndiaFull time

About this role

The role

You will build GenAI-powered features that ship to real users: retrieval-augmented generation (RAG) services, structured data extraction from documents, LLM-backed APIs, and agent-style workflows with human review. You will also write the ordinary engineering that surrounds a model — validation, APIs, data access, tests and monitoring — because that is what makes an AI feature dependable. You will work in a team with a Technical Lead who reviews your designs and code.

This is a hands-on engineering role, not a research or prompt-writing role.

What you will do

  • Build RAG pipelines end to end: document ingestion, chunking, embedding, vector search, re-ranking, grounded answers with citations.

  • Implement structured extraction from documents and semi-structured text into validated JSON, with confidence handling and fallback.

  • Design and consume REST APIs around LLM features; integrate with existing applications and workflow pipelines.

  • Implement tool calling and Agentic workflows with clear guardrails and human-in-the-loop steps where needed.

  • Write evaluation sets and tests for LLM features — measure retrieval quality and answer accuracy, and catch regressions before release.

  • Track cost, latency and failures in production; debug why a request went wrong.

  • Follow good engineering practice: version control, code review, secrets kept out of code, readable and tested code.

  • Use AI coding assistants productively while remaining accountable for what you commit.

What you must bring

  • 1–2 years of hands-on experience building LLM or GenAI applications, with at least one feature that real users have used — not only a notebook or a hackathon demo.

  • Strong Python fundamentals; comfortable with FastAPI, Flask or Django for building APIs.

  • Experience with at least one LLM API (OpenAI, Azure OpenAI, Anthropic, Gemini or open-weight models) and at least one vector store (pgvector, Qdrant, Pinecone, Chroma, Azure AI Search, MongoDB Atlas Vector Search, FAISS).

  • Working understanding of RAG concepts and failure modes: chunking trade-offs, retrieval misses, hallucination, prompt injection, context limits.

  • Structured outputs: function or tool calling, JSON schema, Pydantic validation.

  • Git, basic Docker, and the habit of writing tests.

  • Ability to explain, with reasons, when a rule or a simple lookup is a better choice than an LLM.

Good to have

  • LangChain, LangGraph, LlamaIndex, Semantic Kernel or a similar framework — and the judgment to use them sparingly.

  • OCR and document AI tools (Azure Document Intelligence, Tesseract, PaddleOCR).

  • Evaluation tooling (Ragas, DeepEval, promptfoo) or a harness you built yourself.

  • Cloud experience, Azure preferred; workflow orchestrators (Temporal, Airflow, Prefect).

  • Exposure to enterprise or regulated-industry projects; any of AI-102, AZ-204 or equivalent certifications.



Frequently Asked Questions

Is the salary disclosed for the AI Engineer position at benchmarkit?
The salary for this AI Engineer role at benchmarkit is not publicly listed. Click "Apply Now" to learn more about the compensation package on their official careers page.
Where is the AI Engineer position at benchmarkit located?
This AI Engineer role at benchmarkit is based in Pune, India. The position is listed as on-site or hybrid. Check the full job description or apply directly to confirm the work arrangement.
Is the AI Engineer role at benchmarkit full-time or part-time?
This is listed as a Full time position. It is posted as a AI Engineer role at benchmarkit.
How do I apply for the AI Engineer position at benchmarkit?
Click the "Apply Now" button on this page. You will be redirected to benchmarkit's official application portal hosted on zohorecruit where you can submit your application directly.
When was the AI Engineer job at benchmarkit posted?
This AI Engineer position at benchmarkit was posted on Sep 4, 2026. Apply as soon as possible — early applications are often reviewed first.
AI Engineer
benchmarkit
Apply for this role ↗

You'll be redirected to benchmarkit's official application page on zohorecruit.