AI Agent Developer (LLM Specialist)
Role Overview
The AI Agent Developer is responsible for designing, building, and deploying intelligent autonomous or semi-autonomous agents powered by Large Language Models (LLMs). This role focuses on enabling complex workflows, reasoning capabilities, system integration, and secure data interaction across enterprise environments.
Key Responsibilities
🔹 Agent Design & Development
- Architect and implement AI-powered agents using LLMs (e.g., GPT, Claude, Llama, Mistral, Gemini).
- Develop prompt workflows, retrieval-augmented generation (RAG), and multi-step reasoning.
- Configure agent behaviors for decision-making, task execution, and contextual understanding.
- Implement function calling, tool use, or API integration to extend agent capabilities.
🔹 System Integration
- Connect agents to internal systems, databases, knowledge bases, and APIs.
- Use frameworks such as LangChain, LlamaIndex, Haystack, Semantic Kernel, or custom pipelines.
- Implement connectors for CRMs, ERPs, ticketing platforms, or document repositories.
🔹 Data & Knowledge Management
- Create structured and unstructured knowledge retrieval mechanisms.
- Build and manage vector databases (Pinecone, Weaviate, Chroma, Milvus, Azure AI Search).
- Implement embeddings, chunking strategies, metadata tagging, and query optimization.
🔹 Workflow Automation
- Design agent orchestration pipelines to handle multi-agent collaboration.
- Enable agents to automate research, reporting, email drafting, document processing, and customer interactions.
- Integrate with workflow tools like Zapier, Airflow, UiPath, or Power Automate.
🔹 Model Optimization & Evaluation
- Fine-tune or customize LLMs using organization-specific data.
- Evaluate model performance for accuracy, latency, cost, hallucinations, and compliance.
- Apply guardrails, prompt validation, and reinforcement techniques.
🔹 Security, Ethics & Compliance
- Enforce data privacy and governance standards (GDPR, HIPAA, SOC2, ISO, CCPA).
- Implement role-based access and secure API token handling.
- Apply content filtering, moderation, and usage monitoring.
🔹 Collaboration & Documentation
- Work with cross-functional teams (engineering, design, product, legal, security).
- Maintain technical documentation, agent configuration logs, and deployment playbooks.
- Train internal teams on capabilities and responsible use.
Requirements
Required Skills & Experience
Technical Stack
- Experience with Python, TypeScript/JavaScript, or Go.
- Familiarity with API development (REST, GraphQL, JSON).
- Experience with cloud platforms: Azure OpenAI, AWS Bedrock, GCP Vertex AI, or Hugging Face.
- Knowledge of vector databases and retrieval architectures.
AI/ML Expertise
- Strong understanding of LLM architectures, embeddings, prompt engineering, and RAG.
- Experience with fine-tuning or model adaptation techniques.
- Familiarity with agent frameworks (LangChain Agents, CrewAI, AutoGen, Semantic Kernel).
Bonus Skills
- Experience building chatbots, copilots, or domain-specific assistants.
- Knowledge of event-driven architecture, microservices, or serverless functions.
- Exposure to MLOps or LLMOps tooling.
Benefits
ABOUT THE COMPANY
ClearlyAgile is one of the fastest growing Agile companies in the Tampa Bay area. We foster career growth and are focused on having fun while delivering quality products and services to our clients. With Unlimited Paid Time Off; 10 paid holidays; Medical, Vision and Dental benefits that start on your first day; Share Apprecition Rights; 401(k); paid training certifications; and very competitive compensation plans, ClearlyAgile strives to listen to and invest in its most important asset…its people.
Our Mission: Transform our customer’s businesses using Agile methodologies and principles to help them succeed in a flexible, collaborative, self-organizing and fast-paced environment.
Our Values: We hire and reward based on our core values.
- Show Your Passion - Have passion for what you do. Be an active learner with a commitment to grow. Attend and participate in events!
- Be a Champion of Agile - Spread the word. Back it up with examples, case studies, and data points. Lead by example.
- Teamwork - Be cooperative and collaborative while maintaining a team-first mindset.
- Customer-centric - Always focus on the client and their customers. See the world from their perspectives.
- Quality - Maintain a sense of pride in the quality of your work and the collective work of the team.
- Be a Leader - Empower yourself to make decisions and don’t be paralyzed by fear of failure.
We are an equal opportunity employer and committed to a diverse workforce.
To learn more about us visit our website at www.clearlyagile.com.
