Building AI Agent Applications Series – Assembling your AI agent with the Semantic Kernel

by Contributed | Feb 17, 2024 | Technology

This article is contributed. See the original author and article here.

In the previous series of articles, we learned about the basic concepts of AI agents and how to use AutoGen or Semantic Kernel combined with the Azure OpenAI Service Assistant API to build AI agent applications. For different scenarios and workflows, powerful tools need to be assembled to support the operation of the AI agent. If you only use your own tool chain in the AI agent framework to solve enterprise workflow, it will be very limited. AutoGen supports defining tool chains through Function Calling, and developers can define different methods to assemble extended business work chains. As mentioned before, Semantic Kernel has good business-based plug-in creation, management and engineering capabilities. Through AutoGen + Semantic Kernel, powerful AI agent solutions can be built.

Scenario 1 – Constructing a single AI agent for writing technical blogs

As a cloud advocate, I often need to write some technical blogs. In the past, I needed a lot of supporting materials. Although I could write some of the materials through Prompt + LLMs, some professional content might not be enough to meet the requirements. For example, I want to write based on the recorded YouTube video and the syllabus. As shown in the picture above, combine the video script and outline around the three questions as basic materials, and then start writing the blog.

Note: We need to save the data as vector first. There are many methods. You can choose to use different frameworks for embedded vector processing. Here we use Semantic Kernel combined with Qdrant. Of course, the more ideal step is to add this part to the entire technical blog writing agent, which we will introduce in the next scenario.

Because the AI agent simulates human behavior, when designing the AI agent, the steps that need to be set are the same as in my daily work.

1. Find relevant content based on the question


2. Set a blog title, extended content and related guidance, and write it in markdown


3. Save

We can complete steps 1 and 2 through Semantic Kernel. As for step 3, we can directly use the traditional method of reading and writing files. We need to define these three functions ask, writeblog, and saveblog here. After completion, we need to configure Function Calling and set the parameters and function names corresponding to these three functions.

llm_config={

    “config_list”: config_list,

    “functions”: [

        {

            “name”: “ask”,

            “description”: “ask question about Machine Learning,  get basic knowledge”,

            “parameters”: {

                “type”: “object”,

                “properties”: {

                    “question”: {

                        “type”: “string”,

                        “description”: “About Machine Learning”,

                    }

                },

                “required”: [“question”],

            },

        },

        {

            “name”: “writeblog”,

            “description”: “write blogs in markdown format”,

            “parameters”: {

                “type”: “object”,

                “properties”: {

                    “content”: {

                        “type”: “string”,

                        “description”: “basic content”,

                    }

                },

                “required”: [“content”],

            },

        },

        {

            “name”: “saveblog”,

            “description”: “save blogs”,

            “parameters”: {

                “type”: “object”,

                “properties”: {

                    “blog”: {

                        “type”: “string”,

                        “description”: “basic content”,

                    }

                },

                “required”: [“blog”],

            },

        }

    ],

}

Because this is a single AI agent application, we only need to define an Assistant and a UserProxy. We only need to define our goals and inform the relevant steps to run.

assistant = autogen.AssistantAgent(

    name=“assistant”,

    llm_config=llm_config,

)

user_proxy = autogen.UserProxyAgent(

    name=“user_proxy”,

    is_termination_msg=lambda x: x.get(“content”, “”) and x.get(“content”, “”).rstrip().endswith(“TERMINATE”),

    human_input_mode=“NEVER”,

    max_consecutive_auto_reply=10,

    code_execution_config=False

)
user_proxy.register_function(

    function_map={

        “ask”: ask,

        “writeblog”: writeblog,

        “saveblog”: saveblog

    }

)
with Cache.disk():

    await user_proxy.a_initiate_chat(

        assistant,

        message=“””

            I’m writing a blog about Machine Learning. Find the answers to the 3 questions below and write an introduction based on them. After preparing these basic materials, write a blog and save it.
            1. What is Machine Learning?

            2. The difference between AI and ML

            3. The history of Machine Learning
            Let’s go

    ““”

    )

We tried running it and it worked fine. For specific effects, please refer to：

Scenario 2 – Building a multi-agent interactive technical blog editor solution.

In the above scenario, we successfully built a single AI agent for technical blog writing. We hope that our technology will be more intelligent. From content search to writing and saving to translation, it is all completed through AI agent interaction. We can use different job roles to achieve this goal. This can be done by generating code from LLMs in AutoGen, but the uncertainty of this is a bit high. Therefore, it is more reliable to define additional methods through Function Calling to ensure the accuracy of the call. The following is a structural diagram of the division of labor roles:

Notice

1. 
   

   Admin – Define various operations through UserProxy, including the most important methods.

   
   


2. 
   

   Collector KB Assistant – Responsible for downloading relevant subtitle scripts of technical videos from YouTube, saving them locally, and vectorizing them by extracting different knowledge points and saving them to the vector database. Here I only made a video subtitle script. You can also add local documents and support for different types of audio files.

   
   


3. 
   

   Blog Editor Assistant – When the data collection assistant completes its work, it can hand over the work to the blog writing assistant, who will write the blog as required based on a simple question outline (title setting, content expansion, and usage markdown format, etc.), and automatically save the blog to the local after writing.

   
   


4. 
   

   Translation Assistant – Responsible for the translation of blogs in different languages. What I am talking about here is translating Chinese (can be expanded to support more languages)

   
   

Based on the above division of labor, we need to define different methods to support it. At this time, we can use SK to complete related operations.

Here we use AutoGen’s group chat mode to complete related blog work. You can clearly see that you have a team working, which is also the charm of the agent. Set it up with the following code.

groupchat = autogen.GroupChat(

    agents=[user_proxy, collect_kb_assistant, blog_editor_assistant,translate_assistant], messages=[],max_round=30)

manager = autogen.GroupChatManager(groupchat=groupchat, llm_config={‘config_list’: config_list})

“””

    )

The code for group chat dispatch is as follows:

await user_proxy.a_initiate_chat(

    manager,

    message=“””

            Use this link https://www.youtube.com/watch?v=1qs6QKk0DVc as knowledge with collect knowledge assistant. Find the answers to the 3 questions below to write blog  and save and save this blog to local file with blog editor assistant. And translate this blog to Chinese with translate assistant.

            1. What is GitHub Copilot ?

            2. How to Install GitHub Copilot ?

            3. Limitations of GitHub Copilot
           Let’s go

““”

    )

Different from a single AI agent, a manager is configured to coordinate the communication work of multiple AI agents. Of course, you also need to have clear instructions to assign work.

You can view the complete code on this Repo.

If you want to see the result about English blog, you can also click this link. 

If you want to see the result about Chinese blog, you can also click this link. 

More

AutoGen helps us easily define different AI agents and plan how different AI agents interact and operate. The Semantic Kernel is more like a middle layer to help support different ways for agents to solve tasks, which will be of great help to enterprise scenarios. When AutoGen appears, some people may think that it overlaps with Semantic Kernel in many places. In fact, it complements and does not replace it. With the arrival of the Azure OpenAI Service Assistant API, you can believe that the agent will have stronger capabilities as the technical framework and API are improved.

Resources

1. Microsoft Semantic Kernel https://github.com/microsoft/semantic-kernel


2. Microsoft Autogen https://github.com/microsoft/autogen


3. Microsoft Semantic Kernel CookBook https://aka.ms/SemanticKernelCookBook


4. Get started using Azure OpenAI Assistants.  https://learn.microsoft.com/en-us/azure/ai-services/openai/assistants-quickstart


5. What is an agent?  https://learn.microsoft.com/en-us/semantic-kernel/agents


6. What are Memories? https://learn.microsoft.com/en-us/semantic-kernel/memories/

Rapidly scope NC2 on Azure using Nutanix Sizer

by Contributed | Feb 16, 2024 | Technology

This article is contributed. See the original author and article here.

Overview

A global enterprise wants to migrate thousands of Nutanix AHV or VMware vSphere virtual machines (VMs) to Microsoft Azure as part of their application modernization strategy. The first step is to exit their on-premises data centers and rapidly relocate their legacy application VMs to the Nutanix Cloud Clusters on Azure (NC2 on Azure) service as a staging area for the first phase of their modernization strategy. How can they quickly size NC2 on Azure to meet their workload requirements?

NC2 on Azure is a third-party Azure service from Nutanix that provides private clouds containing Nutanix AHV clusters built from dedicated bare-metal Azure infrastructure. It enables customers to leverage their existing investments in Nutanix skills and tools, allowing them to focus on developing and running their Nutanix-based workloads on Azure.

In this post, I will introduce the typical customer workload requirements, describe the NC2 on Azure architectural components, and describe how to use Nutanix Sizer to quickly scope an NC2 on Azure solution.

In the next section, I will introduce the typical sizing requirements of a customer’s workload.

Customer Workload Requirements

A typical customer has multiple application tiers that have specific Service Level Agreement (SLA) requirements that need to be met. These SLAs are usually named by a tiering system such as Platinum, Gold, Silver, and Bronze or Mission-Critical, Business-Critical, Production, and Test/Dev. Each SLA will have different availability, recoverability, performance, manageability, and security requirements that need to be met.

For the initial sizing, customers will have CPU, RAM, Storage and Network requirements. This is normally documented for each application and then aggregated into the total resource requirements for each SLA. For example:

Table 1 – Typical Customer SLA requirements for Performance

The concepts introduced in Table 1 have the following definitions:

• CPU: CPU model and speed (this can be important for legacy single threaded applications), number of cores, vCPU to physical core ratios.


• Memory: Random Access Memory size, Input/Output (I/O) speed and latency, oversubscription ratios.


• Storage: Capacity, Read/Write Input/Output per Second (IOPS) with Input/Output (I/O) size, Read/Write Input/Output Latency, RAID policy, RF policy.


• Network: In/Out Speed, Network Latency (Round Trip Time).

A typical legacy business-critical application will have the following application architecture:

• Load Balancer layer: Uses load balancers to distribute traffic across multiple web servers in the web layer to improve application availability.


• Web layer: Uses web servers to process client requests made via the secure Hypertext Transfer Protocol (HTTPS). Receives traffic from the load balancer layer and forwards to the application layer.


• Application layer: Uses application servers to run software that delivers a business application through a communication protocol. Receives traffic from the web layer and uses the database layer to access stored data.


• Database layer: Uses a relational database management service (RDMS) cluster to store data and provide database services to the application layer.

The application can also be classified as OLTP or OLAP, which have the following characteristics:

• Online Transaction Processing (OLTP) is a type of data processing that consists of executing several transactions occurring concurrently. For example, online banking, retail shopping, or sending text messages. OLTP systems tend to have a performance profile that is latency sensitive, choppy CPU demands, with small amounts of data being read and written.


• Online Analytical Processing (OLAP) is a technology that organizes large business databases and supports complex analysis. It can be used to perform complex analytical queries without negatively impacting transactional systems (OLTP). For example, data warehouse systems, business performance analysis, marketing analysis. OLAP systems tend to have a performance profile that is latency tolerant, requires large amounts of storage for records processing, has a steady state of CPU, RAM and storage throughput.

Depending upon the requirements for each service, the infrastructure design could be a mix of technologies used to meet the different application SLAs with cost efficiency.

Figure 1 – Typical Legacy Business-Critical Application Architecture

In the next section, I will introduce the architectural components of the NC2 on Azure service.

Architectural Components

The diagram below describes the architectural components of the NC2 on Azure service.

Figure 2 – NC2 on Azure Architectural Components

Each NC2 on Azure architectural component has the following function:

• Azure Subscription: Used to provide controlled access, budget, and quota management for the NC2 on Azure service.


• Azure Region: Physical locations around the world where we group data centers into Availability Zones (AZs) and then group AZs into regions.


• Azure Resource Group: Container used to place Azure services and resources into logical groups.


• NC2 on Azure: Uses Nutanix software, including Prism Central, Prism Element, Nutanix Flow software-defined networking, Nutanix Acropolis Operating System (AOS) software-defined storage, and Azure bare-metal Acropolis Hypervisor (AHV) hosts to provide compute, networking, and storage resources.


• Nutanix Move: Provides migration services.


• Nutanix Disaster Recovery: Provides Disaster Recovery automation and storage replication services.


• Nutanix Files: Filer services.


• Nutanix Objects: Object storage services.


• Nutanix Self Service: Application Lifecycle Management and Cloud Orchestration.


• Nutanix Cost Governance: Multi-Cloud Optimization to reduce cost & enhance Cloud Security.


• Azure Virtual Network (VNet): Private network used to connect Azure services and resources together.


• Azure Route Server: Enables network appliances to exchange dynamic route information with Azure networks.


• Azure Virtual Network Gateway: Cross premises gateway for connecting Azure services and resources to other private networks using IPSec VPN, ExpressRoute, and VNet to VNet.


• Azure ExpressRoute: Provides high-speed private connections between Azure data centers and on-premises or colocation infrastructure.


• Azure Virtual WAN (vWAN): Aggregates networking, security, and routing functions together into a single unified Wide Area Network (WAN).

In the next section, I will describe how to use the Nutanix Sizer to quickly scope the NC2 on Azure service for a customer workload.

Using the Nutanix Sizer

The Nutanix Sizer is available to Nutanix Employees and Nutanix Partners. If you are a Nutanix Customer, please reach out to your Nutanix, Microsoft, or Partner account team to engage an architect to size your NC2 on Azure solution. Customers also have access to Nutanix Sizer Basic.

Unless specified, all other settings can be left at the default values. Once the scenario is built, it can be later tweaked to meet the customer requirements.

Step 1: Access My Nutanix and select the Nutanix Sizer Launch button.

Figure 3 – My Nutanix Dashboard

Step 2: Select the Create Scenario button.

Figure 4 – Nutanix Sizer My Scenarios

Step 3: Enter the Scenario Name, Install Country, and select the Create button.

Figure 5 – Nutanix Sizer Create New Scenario

Optionally, if you have a good understanding of the problem the customer is trying to solve, you can fill out the Scenario Objectives (Executive Summary, Requirements, Constraints, Assumptions, and Risks) to start building out the design. This will also allow you to use the advanced export features at the end of the sizing process.

Step 4: Press the Add button in the Create Workloads pane. If you want Import a Nutanix Collector or RVTools files as the source file for workload, select the Import button instead.

Figure 6 – Nutanix Sizer Create Workloads

Step 5: Define the Workload Name, Workload Type, Server Profile Size, and Number of VMs. Then select the Save & Review Cluster button.

Figure 7 – Nutanix Sizer Add Workload

Step 6: Select NC2 on Azure from the Vendor section of the Platform Settings. Then scroll down to the Cluster Settings.

Figure 8 – Nutanix Sizer Platform Settings

Step 7: Select the Environment Type from the Cluster Settings and press the Apply button.

Figure 9 – Nutanix Sizer Cluster Settings

Step 8: In the Workloads Summary page, select the Solution tab.

Figure 10 – Nutanix Sizer Workloads Summary

Step 9: In the Solution Summary page, verify the NC2 on Azure tag is present in each cluster.

Figure 11 – Nutanix Sizer Solution Summary

Step 10: In the Solution Summary page, scroll down to the Sizing Details for the detailed breakdown.

Figure 12 – Nutanix Sizer Solution Sizing Details

Step 11: To share the Scenario with others:

• Select BOM, Download BOM


• Select Quote, Generate Budgetary Quote or Generate Frontline Quote


• Select More, Share Scenario or Create Proposal

Figure 13 – Nutanix Sizer Export & Sharing Options

In the following section, I will describe the next steps that need to be made to progress this high-level design estimate towards a validated detailed design.

Next Steps

The NC2 on Azure sizing estimate has been assessed using Nutanix Sizer. With large enterprise solutions for strategic and major customers, a Nutanix Solutions Architect from Azure, Nutanix, or a trusted Nutanix Partner should be engaged to ensure the solution is correctly sized to deliver business value with the minimum of risk. This should also include an application dependency assessment to understand the mapping between application groups and identify areas of data gravity, application network traffic flows, and network latency dependencies.

Summary

In this post, we took a closer look at the typical sizing requirements of a customer workload, the architectural building blocks, and the use of Nutanix Sizer to quickly scope the NC2 on Azure service. We also discussed the next steps to continue an NC2 on Azure design.

If you are interested in NC2 on Azure, please use these resources to learn more about the service:

• Homepage: Elevate Azure Clusters with Nutanix


• Documentation: What is BareMetal Infrastructure for Nutanix Cloud Clusters on Azure?


• Use-cases: Use cases and supported scenarios


• Solution Design: Solution design – Azure Baremetal Infrastructure


• Architecture: Architecture of BareMetal Infrastructure for NC2


• Getting Started: Microsoft Azure Marketplace


• Solution Requirements: Requirements – Azure Baremetal Infrastructure


• Azure Regions: Supported instances and regions


• SKUs: SKUs – Azure Baremetal Infrastructure


• User Guide: Nutanix Cloud Clusters on Azure Deployment and User Guide


• Nutanix Sizer: Sizer 6.0 User Guide


• FAQ: FAQ – Azure Baremetal Infrastructure

Author Bio

René van den Bedem is a Principal Technical Program Manager at Microsoft. His background is in enterprise architecture with extensive experience across all facets of the enterprise, public cloud & service provider spaces, including digital transformation and the business, enterprise, and technology architecture stacks. René works backwards from the problem to be solved and designs solutions that deliver business value with the minimum of risk. In addition to being the first quadruple VMware Certified Design Expert (VCDX), he is also a Dell Technologies Certified Master Enterprise Architect, a Nutanix Platform Expert (NPX), an NPX Panelist, and a Nutanix Technology Champion.