Agent = GenAI + Tools + Memory (Data) interacting with external env.
AI Agent = system that integrates Perception, Planning, Tools, Actions + an interface to interact with an external environment (a user or another ecosystem) to achieve a goal in it. Proper agent design depends on task - e.g. medical agent’s errors can be fatal => agent can suggest Human in the Loop (doctor).
Perception of input form external env - chatbot user w/prompts OR patient (lab, vitals, imaging) => decide what’s relevant or missing. Persona may be defined.
Planning & reasoning - LLM autonomously analyzes goal through inputs & context => determines a sequence of steps / tools to accomplish a complex task that can't be completed in a single action. Example: online research - break into subtopics, synthesize findings, compile a report. Methods: CoT, ReAct, recall important memories, decide which tools are needed, how to synthesize later on. Two regimes: interleaved (ReAct - decide the next step after each observation; adaptive, more LLM calls) vs. plan-then-execute (draft full plan up front, execute steps, replan on failure; cheaper, auditable, parallelizable). Tree of Thoughts: branch into multiple reasoning paths, evaluate, backtrack — search over reasoning.
Take actions that change the environment towards the goal. (call API/KBs, outputs a response).
Tool Use - LLM performs complex, multi-step tasks efficiently by leveraging up to hundreds of external tools / APIs to gather info, take actions, or manipulate data, extending its capabilities. Agent decides which tool to use: prior memories, calculator API for math, weather API, search different sources (web, Wikipedia, arXiv, KB/KG etc.), interface w/productivity tools (email, read/write calendar, etc.) OR execute code. Modular & extensible - add more tools and add them to prompt, but requires robust tool error handling & security risk when using external services (PII). Rules: output json, use calc if math, weather tool if weather, tool outputs code - run it.
Typical agentic workflows / patterns :
Reflection: automates critical feedback => model auto-criticizes its output constructively and improves the response. LLM can have tools to evaluate its output (run unit tests) or search web to double-check its output.
Multi-agent collaboration: decomposing complex tasks into subtasks, each handled by different agents, e.g. writing software => helps LLMs focus on smaller tasks. Different agents prompt one or more LLMs for specific tasks. Even if same LLM - different focused optimized prompts.
Hierarchical - Agent Orchestration - central manager agent receives the task, decomposes it, delegates subtasks to worker agents, then aggregates results, and decides when the task is done. Workers don't talk to each other Need fine-tuning or few shot examples for the manager.
Decentralized collaboration. No fixed manager — agents interact directly, handing off or debating. Sub-patterns worth naming: sequential/pipeline (writer → reviewer → editor, each output feeding the next), handoff/network (any agent can transfer control to any other — OpenAI's Swarm/Agents SDK popularized this), debate/reflection (agents critique each other's outputs to improve quality — generator-critic), and blackboard (agents share a common workspace/state rather than messaging directly).
ReAct Pattern - alternation: REAson, take ACTion (tool/API call, access memory, respond, etc.), reason again, repeat until answering user query. Multi-step tasks, integration w/external systems/APIs, but complex design, need state management and error handling, slower.
Memory (Retrieval) Augmented Pattern - one of tools is DB or info storage to extract context-relevant info to augment prompt. Order favorite pizza - what’s favorite, credit card # from past orders, etc. Personalization, learning, but needs memory management & retrieval strategies, risk of outdated memory, PII.
Human in the Loop - like ReAct, but a human approves the action before it’s taken. Agent learns from its mistakes. Agent can bypass human if confidence is high.
Program-Aided Language Model - translate a task into Python code
Chain of Thought (prompting technique) - decompose task into reasoning steps (each step - LLM call for its own reasoning). Multi-step tasks, easier debugging, but requires careful agent design. Prompt: Work in these phases: plan, solve, critique, finalize. For each step output: {phase…, content…}.
Use Cases :
Finance: Auto-trading, risk management, fraud detection.
Healthcare: Diagnostics, patient management, personalized medicine.
Manufacturing: Production optimization, supply chain management, predictive maintenance.
Education: Personalized learning, administrative automation, intelligent tutoring.
Publishing: Content creation, editorial processes, recommender systems.
Types :
1. Creative Engines: new content, creativity.
2. Information Retrievers: extract info from DBs, search engines, APIs.
3. Syntactic Operations: grammar correction, rephrasing, summarization, translation.
4. Logic Engines: break down complex tasks into logical steps and create action plans.
More Agents Is All You Need: more agents increase LLMs accuracy (sampling-and-voting technique). E.g. with 15 agents, Llama2-13B equals Llama2-70B.