6 types of AI Agents

With the development of large models, general intelligence is constantly iterating and upgrading, and the application mode is constantly innovating, from simple prompt applications, RAG (search enhancement generation) to AI Agent (artificial intelligence agent). Among them, AI Agent has always been a hot topic and will be everywhere in the future. Bill Gates also claimed that the ultimate technology competition will revolve around the development of top AI agents. He said: "You will never go to search websites or Amazon again..." This shows that he is optimistic about the huge changes that artificial intelligence will bring to the human-computer interaction mode, and also recognizes the important role played by AI Agent in it.

AI Agent is a virtual assistant driven by artificial intelligence that can help automate processes, generate insights, and improve efficiency. It can act as an employee or partner to help achieve human-given goals.

A simple example of an AI agent is a thermostat that adjusts the heating to a specific temperature based on a specific time. It senses the environment through a temperature sensor and a clock. It takes action through a switch that turns the heating on or off based on the actual temperature or time. The thermostat can be turned into a more complex AI agent by adding AI capabilities that enable it to learn from the habits of the people living in the house.

AI Agents can be divided into different types based on how their behaviors affect perceived intelligence and capabilities.

This article mainly introduces 6 different AI Agents. Including:

• Simple reflex agents
• Model-based agents
• Goal-based agents
• Utility-based agents
• Learning agents
• Hierarchical agents

1. Simple reflex agents: Simple reflex agents

A simple reflex agent is an AI system that can make decisions based on predefined rules. It only reacts to the current situation without considering past or future consequences.

A simple reflexive agent is suitable for environments with stable rules and straightforward actions, because its behavior is purely reactive and responds instantly to changes in the environment.

(1) Principle:

Simple reflex agents perform their functions by following conditional,action rules, which specify the actions to be taken under certain conditions.

(2) Example:

A rule-based system for implementing intelligent customer service. If a customer's message contains the keyword "password reset", the system can automatically generate a predefined response containing instructions on resetting the password.

(3) Advantages:

• Simplicity: Easy to design and implement, requires few computing resources, and does not require extensive training or complex hardware.
• Implementation: Ability to respond to environmental changes in real time.
• High reliability: High reliability when the sensors providing the input are accurate and the rules are well designed.

(4) Weaknesses:

• This can be prone to error if the input sensors are faulty or the rules are poorly designed.
• There is no memory or state, which limits their applicability.
• Unable to handle changes in the environment that were not explicitly programmed.
• Limited to a specific set of operations and unable to adapt to new situations.

2. Model-based agents: Model-based agents

The model-based agent performs actions based on the current perception and an internal state representing the unobservable words. It updates its internal state based on two factors:

• How the world evolves independently of agents
• How Agents' Actions Affect the World

(1) Principle:

Model-based agents follow condition+action rules that specify the appropriate action to take in a given situation. But unlike simple reflexive agents, model-based agents also use their internal state to evaluate conditions during decision and action processes.

The model-based agent operates in four phases:

• Perception: It senses the current state of the world through sensors.
• Model: It builds an internal model of the world based on what it sees.
• Reason: It uses its own model of the world to decide how to act based on a set of predefined rules or regulations.
• Behavior: The agent performs the actions of his or her choice.

(2) Example:

https://aws.amazon.com/cn/bedrock/

One of the best examples of model-based agents is Amazon Bedrock, a service that uses underlying models to simulate operations, gain insights, and make informed decisions for effective planning and optimization.

Through various models Bedrock can gain insights, predict outcomes and make informed decisions. It continuously improves its models using real data, allowing it to adapt and optimize its operations.

Amazon Bedrock then plans for different scenarios and selects the best strategy by simulating and tuning model parameters.

(3) Advantages:

• Make fast and effective decisions based on your understanding of the world.
• Better able to make accurate decisions by building an internal model of the world.
• Adapt to environmental changes by updating internal models.
• By using its internal state and rules to determine the condition.

(4) Weaknesses:

• The computational cost of building and maintaining models can be high.
• These models may not capture the complexity of real-world environments well.
• Models cannot predict all potential situations that may arise.
• Models need to be updated frequently to stay current.
• Models may have challenges in terms of understanding and interpretability.

3. Goal-based agents: Goal-based agents

Goal-based agents are artificial intelligence agents that use information from their environment to achieve a specific goal. They use search algorithms to find the most efficient path to achieve a goal in a given environment.

These agents are also called rule-based agents because they follow predefined rules to achieve goals and take specific actions based on certain conditions.

Goal-based agents are easy to design and can handle complex tasks. They can be used in various applications such as robotics, computer vision, and natural language processing.

Unlike basic models, goal-based agents can determine the best path for decision-making and action courses based on their desired outcomes or goals.

(1) Principle:

Given a plan, a goal-based agent attempts to select the best strategy to achieve the goal and then uses a search algorithm to find an efficient path to the goal.

The working mode of goal-based proxy can be divided into five steps:

• Perception: An agent perceives its environment using sensors or other input devices to gather information about its surroundings.
• Reasoning: The agent analyzes the information it has collected and decides on the best course of action to achieve its goal.
• Action: The agent takes actions to achieve its goal, such as moving or manipulating objects in the environment.
• Evaluation: After taking an action, the agent evaluates its progress toward its goal and adjusts its actions if necessary.
• Goal Completion: Once the agent achieves its goal, it either stops working or starts working on a new goal.

(2) Example:

https://blog.google/technology/ai/bard-google-ai-search-updates/

Google Bard is a learning agent. In a sense, it is also a goal-based agent. As a goal-based agent, its goal is to provide high-quality responses to user queries. The actions it chooses are likely to help users find the information they need and achieve their intended goal of getting accurate and useful responses.

(3) Advantages:

• Easy to understand and implement.
• Effectively achieve specific goals.
• Easy to evaluate performance against goal achievement.
• It can be combined with other AI techniques to create more advanced agents.
• Ideal for well-defined, structured environments.
• It can be used in various applications such as: robotics, gaming, and self-driving cars.

(4) Weaknesses:

• Limited to specific goals.
• Unable to adapt to changing circumstances.
• Not effective for complex tasks with too many variables.
• Extensive domain knowledge is required to define the goals.

4. Utility-based agents: Utility-based agents

Utility-based agents are AI agents that make decisions based on utility functions or value maximization. They choose actions with the highest expected utility, and the outcome of this choice determines the final outcome. This model is more flexible and adaptive to handle tasks in complex situations.

Utility-based agents are often used when comparisons and choices must be made among multiple options, such as how resources are allocated, how tasks are scheduled, or how a game is played.

(1) Principle:

• A utility-based agent aims to choose actions that lead to high-utility states. To achieve this, it needs to model its environment, which can be either simple or complex.
• Then, the expected utility of each possible outcome is evaluated based on the probability distribution and the utility function.
• Finally, the action with the highest expected utility is chosen and this process is repeated at each time step.

(2) Example:

https://www.anthropic.com/news/introducing-claude

 Anthropic Claude is an AI tool whose goal is to help cardholders maximize the rewards they get from using their cards and is a utility-based agent.

To achieve its goal, it uses a utility function that assigns numerical values ​​representing success or happiness to different states (situations faced by the cardholder, such as: making a purchase, paying a bill, redeeming a reward, etc.) It then compares the results of different actions in each state and weighs the decisions based on their utility values.

Additionally, it uses heuristics and artificial intelligence techniques to simplify and improve decision making.

(3) Advantages:

• Can handle a wide range of decision-making problems
• Learn from experience and adjust their decision-making strategies
• Provide a unified and objective framework for decision-making applications

(4) Weaknesses:

• An accurate model of the environment is required, otherwise it will lead to wrong decisions
• High computational cost, requiring a lot of calculations
• No consideration of moral or ethical factors
• It is difficult for humans to understand and verify its process

5. Learning agents

A learning agent is a model that can learn from past experience and improve model performance. The initial agent has basic knowledge and continues to grow through machine learning and automatic adaptation.

The learning agent consists of four main components:

• Learning Element: It is responsible for learning and making improvements based on the experience gained from the environment.
• Citric: It provides feedback to the learning element through the agent’s performance to predefined criteria.
• Performance Element: It selects and executes external actions based on information from the Learning Element and the Critic.
• Question Generator: It suggests actions to create new information experiences for learning elements to improve their performance.

(1) Principle:

AI learning agents follow a closed loop of observation, learning, and action based on feedback. They interact with the environment, learn from feedback, and modify their behavior for future interactions.

Here’s how this closed loop works:

• Observation: The learning agent observes its environment through sensors or other inputs.
• Learning: Agents analyze data using algorithms and statistical models and learn from feedback on their behavior and performance.
• Action: Based on what it has learned, the agent takes actions in its environment to decide how to act.
• Feedback: Agents receive feedback about their actions and performance through rewards, penalties, or environmental cues.
• Adaptation: Using feedback, the agent changes its behavior and decision-making process, updates its knowledge and adapts to its environment.
• This cyclical process repeats over time, enabling the agent to continuously improve its performance and adapt to changing circumstances.

(2) Example:

https://dataconomy.com/2023/04/13/what-is-autogpt-and-how-to-use-ai-agents/

AutoGPT is a great example of a learning agent, let’s say you want to buy a smartphone. So, you give AutoGPT a prompt to do a market research on the top 10 smartphones and provide insights about their pros and cons.

To complete your task, AutoGPT will analyze the pros and cons of the top 10 smartphones by exploring various websites and sources. It uses sub-agents to assess the authenticity of the websites. Finally, it generates a detailed report summarizing the findings and listing the pros and cons of the top 10 smartphone companies.

(3) Advantages:

• Agent can turn thoughts into actions based on artificial intelligence decisions
• Learning agents can follow basic commands, such as verbal instructions, and perform tasks
• Unlike classical agents that perform predefined actions, learning agents can evolve over time.
• AI agents take utility measurements into account, making them more realistic

(4) Weaknesses:

• May lead to biased or incorrect decision making
• High development and maintenance costs
• Requires a lot of computing resources
• Reliance on large amounts of data
• Lack of human intuition and creativity

6.Hierarchical agents: Hierarchical agents

A hierarchical proxy is a hierarchical structure that can contain high-level agents and low-level agents, with high-level agents supervising low-level agents. However, these levels may vary depending on the complexity of the system.

Application scenarios of hierarchical agents include: robotics, manufacturing, transportation, etc. It is good at coordination, handling multi-tasks and sub-tasks.

(1) Principle:

Hierarchical agents work like a company’s organization. They organize tasks in a structured hierarchy consisting of different levels, where higher-level agents oversee and break down goals into smaller tasks.

Lower-level agents then perform these tasks and provide progress reports.

In the case of complex systems, there may be intermediate level agents that coordinate the activities of lower level agents with higher level agents.

(2) Example:

https://research.google/blog/unipi-learning-universal-policies-via-text-guided-video-generation/

Google’s UniPi is an innovative AI hierarchical agent that leverages text and video as a universal interface, enabling it to learn a variety of tasks in a variety of environments.

UniPi consists of a high-level policy that generates instructions and demonstrations and a low-level policy that executes tasks. The high-level policy adapts to various environments and tasks, while the low-level policy learns through imitation and reinforcement learning.

This hierarchical structure enables UniPi to effectively combine high-level reasoning and low-level execution.

(3) Advantages:

• Hierarchical agents provide resource efficiency by assigning tasks to the most appropriate agent and avoiding duplication of work.
• Hierarchical structures enhance communication by establishing clear lines of authority and direction.
• Hierarchical reinforcement learning (HRL) improves agent decision making by reducing action complexity and enhancing exploration. It adopts high-level operations to simplify the problem and facilitate agent learning.
• Hierarchical decomposition provides the benefit of minimizing computational complexity by providing a more concise and reusable representation of the entire problem.

(4) Weaknesses:

• Complexity arises when using hierarchies to solve problems.
• Fixed hierarchies limit adaptability in changing or uncertain environments, hindering the agent’s ability to adjust or find alternatives.
• Hierarchical proxies follow a top-down control flow, which can cause bottlenecks and delays even when lower-level tasks are ready.
• Hierarchies may lack reusability across different problem domains, requiring the creation of new hierarchies for each domain, which is time consuming and dependent on expertise.
• Training hierarchical agents is challenging due to the need for labeled training data and elaborate algorithm design. Applying standard machine learning techniques to improve performance is even more difficult due to their complexity.

Summarize

With the recent rapid iteration and upgrade of large language models, AI agents are no longer a new thing. When we put multiple agents together, the agent capabilities of a team will far exceed that of a single agent. From simple reflex agents that maintain the temperature of a home to more advanced agents that drive a car, AI agents will be everywhere. In the future, it will be easier for everyone to create their own agent and their own agent team. It enables people to complete tasks that may take hours or days in minutes!