Microsoft’s Autogen 2 – Create Custom AI Agents
Microsoft’s Autogen 2 – Create Custom AI Agents is a deep dive video on the newly released Autogen version 2 by Microsoft. This upgraded software allows users to interact and use AI agents on their devices with ease. The video focuses on how to use Autogen Studio, the second iteration of Microsoft Autogen, to build multi-agent apps. The software enables users to declaratively define and modify agents and multi-agent workflows through a user-friendly interface. By selecting parameters and adding skills to agents, users can accomplish various tasks efficiently. Autogen Studio is open source and can be easily installed via pip. The video also guides users through the installation process, including configuring a language model provider and setting up an API key. The video showcases the user interface of Autogen Studio, organized into three sections – build, playground, and gallery, demonstrating how users can create custom AI agents and work on various tasks seamlessly. For those facing any issues during installation, the video recommends using GPT 4 for troubleshooting.
Watch the video by TheAIGRID
Video Transcript
So Microsoft recently released autogen version 2 and this is essentially the upgraded version of Microsoft autogen and auto Jun was one of the first Microsoft softwares that actually allowed you to interact and use AI agents on your device now essentially this video is going to be a deep dive on
How to use autogen Studio which is the second iteration of Microsoft autogen how you can build multi-agent apps using this software don’t forget to check some of the links down below if you are struggling and let’s take a look at exactly how Microsoft autogen Works so essentially by this screenshot you can
See that this is the autogen studio and later on in the video I’ll be showing you some of the actual things that you can really really do so essentially they say that to rapidly help you prototype multi-agent solutions for your tasks we are introducing autogen Studio an interface powered by autogen it allows
You to declaratively Define and modify agents and multi-agent workflows through through point and click and drag and drop interface and you for example you can select the parameters of two agents that will communicate to solve your task you can use our user interface to create chat sessions with the specified agents
And view the results for example view chat history generated files and the time taken then of course you can explicitly add skills to your agent and accomplish more tasks and you can also publish your sessions to a local gallery now autogen studio is open source and can be installed via pip so let’s
Actually take a look quickly at the installation process it’s actually fairly simple and if you do have any issues I have a One-Stop solution which I will show you all so you can see right here the first thing that you need to do is configure a large language model
Provider it says here to get started you need to access a language model you can set this up by following the steps in the autogen documentation and config and configure your environment with either the open AI key or an azur openi key now essentially it’s just an API key so if
You want to get yourself an API key if you want to get yourself an API key in order to use this what you need to do is you need to head on over to this page so essentially all you want to do is you just want to sign in to the open AI
Playground area and what you want to be able to make sure you do is you then scroll down to here then you just click API Keys then you can see here that there are many different API keys that you can use and can create now I’ve created a couple of API keys and
Essentially when you create a new key here it’s going to tell you exactly what you need to do so if I click create a new key here um you just name it as whatever you want so I could name this as autogen um studio and then for
Example here I could click create key then of course I have to verify the puzzle and I’m not going to lie to you guys sometimes these puzzles are actually pretty hard like um there was one previously that I did have to do that was really really hard but um you
Know you just do the puzzle or whatever and then you do get your API key so I just wait for here and then I get my API key now I’m not going to show you guys what the key is but essentially um you just copy it you copy it and then of
Course just put that into a text document so that you keep keep that safe because once you do create the key you’re going to be able to have to generate a new one because you’re not going to be able to view this key again so once you do that just copy that done
And then um yeah so as you can see autogen studio and then um last use never and then of course I can just um revoke the key just delete it or I can you know edit the key um just by editing the name of that key right there so once
You have that key um then we go back to this and then you can see you can also specify the model directly in the agent configuration as shown below I would I would just rather not do that but then of course um in the terminal this is
Where you set the open AI API key so now of course we have the installation which is right here so essentially all you want to do is open up the command prompt and then all you want to do is literally just type in PIP install autogen studio
If you actually go onto the autogen studio page you can see that the text is actually right here so if you just copy this and then once you copy this if you actually open up your command prompt and then this is the command prompt and then
You just put that in there just put in/ pip install now I’ve already in I’ve already installed it already so I don’t need to do this but this is something that you do want to do if you do want to install it then of course this is
Running the application so I do have it running but once you do have it installed and once you do have um the application ready all you need to do is just literally copy the text like I said a link will be in the description so it’ll be really easy to copy this and
Then you just simply copy that put it into your command prompt terminal and then it will essentially give you a link now the link is usually here so once you see the https a local host all you need to do is simply do that so I’m going to
Do that again just to show you guys how it works so I’ve typed in Port Y and then of course it’s running and then you can see here that it gives me the link so I just copy this right here uh just get that then contrl + C and it says uh
Waiting for application startup application startup complete running on y y y then all I do is I literally just paste that into a browser and you can see that autogen studio is loaded up now if you are having any trouble with this cuz I actually did have a few you know
Troubles and a a few niggles when I actually started this out um I would just you know diagnose it with GPT 4 so if it says any kind of error just get the error paste it into GPT 4 and just tell them look I’m trying to install
This program so I just copied and pasted the entire page and I was like I’m trying to install this but I’ve have encountered this error and just copy and paste it in and gbt 4 will solve it like it did for me so that is something that
Actually does help me quite a bit so then of course we have the actual use cases of what you can actually do with autogen studio so the autogen studio UI is organized into three high level sections which are build playground and gallery so I’m going to you know firstly
Walk you through then I’m going to show you guys how to use everything so first of all we have the build so essentially what build is is really simple because this is where you can actually do your general agent workflow um and this is where you have your general agents and
You can essentially like build them and stuff like that but like I said these are just screenshots I’m going to show you guys in a second and of course we have the skills area so skills are basically the functions that describe how to solve a task and in general a
Good skill has a descriptive name such as you know for example generate images extensive doc strings and good defaults for example writing out files to dis for persistence and reuse and you can add new skills to autogen Studio that are via the provided user interface and at
That and at inference time these skills are made available to the assistant agent as they address your tasks and of course here we have the agent workflows which is just a specification of a set of agents that can work together to accomplish the task and the simplest
Version of this is a setup with two agents a user proxy agent that represents for example it compiles code and prints a result and an assistant that can address task requests for example generating plans writing code evaluating responses and closing error recovery steps and a more complex flow
Could be a group chat where even more agents Works towards a solution so for example here like you guys can see we’ve got the build playground and gallery and for mine currently there is a so here you can see we’ve got build we’ve got playground and we’ve got gallery and you
Can see right here this autogen studio build multi-agent apps and you can see here that these are the default workflow so we’ve got the visualization agent workflow so this workflow is used for visualization tasks and it says here that visualization assistant to create plans and to write code and then we can
See the system message right here which we can actually read it says your task is to ensure you create highly a high quality visualization for the user your visualizations must follow bre’s practices and you must articulate your reasoning or your choices visualization must not have a grid outliner box the
Visualization should have an appropriate aspect ratio for example rectangular for time series data the title must be Bly y y yada if the chart is a line chart and basically this is um how we have our system message then of course here we have our models and then of course there
Are skills so we can you know have different skills here so for example one skill that we have is find papers from arxiv I’m not even sure how you say that but you can see right here it uses the arive API in order to find certain
Papers so one thing that you could do is you could um get it to research papers and this is going to be a skill that it does have of course generate images I’m presuming that it does use doly 3 to generate images here um this something
That I’m going to show you guys how to do as well um and you can see right here response client generate images model darly 3 so this is another skills you can give you know your agent uh different images which is really cool um I mean not different images different um
Skills which is really cool and of course we’ve got fetch profile here which is you know you can see right here you know fetch different and so yeah this is fetch profile I’m not actually sure what this is but it says fetches the text content from a personal website
Given the URL yada yada y this function scrapes the page and Returns the text found into the body so um yeah that’s actually what it does um and you can actually add different skills I’m guessing that you can create new skills I’m I’m guessing that you probably um
Use GPT 4 to create new skills um and then of course you can see here a general agent workflow and this is a primary assistant default assist to generate plans and write code you see um helpful assistant user yada y y then of course um we can make new different
Workflows then of course we can um have our we have our agents here so configure an agent that can be rused in your workflow um and essentially how this works is that you can see that we have different agents and then different workflows so this workflow um right here
You can see is the general agent workflow so so we have the sender and then of course we have the receiver then of course we have here we have the sender and then of course we have the receiver and then of course we add a new
Workflow um then of course here we have the skills and if you want to add the new skills um I’m sure you’re going to have to generate the code maybe you could ask gb4 to do it this is a sample skill replace with your own skill function I’m not exactly the very best
At coding so I don’t know exactly how to do this but I’m sure there are some other guides that I will leave linked down in the description below but it does actually work so um now I’m going to show you guys some of the demos of
This stuff in action so this is from a different session where I was loading this on a different device and I will leave um instructions on how to actually uh do this like a full list in the description or perhaps it may actually be on the a grid website because the
Description is actually pretty pretty extensive so essentially here you can see that this is autogen and essentially I’m going to walk you through exactly what was going on so here you can see that I’m looking at the skills fetching profile and the different you know skills that you know the each agent is
Able to have you’re able to just add different skills skills so this is a sample skill that I added um this was generating an asy art of an cat which is essentially just a drawing of a cap using um text so um essentially here you can see that the skill was retrieved
Successfully and that skill was there then of course once that is done you can see I’m looking over the agent specification I’m just making sure that everything is good just outlining you know the models all the skills you can see you can find papers on arive generate images um and all that good
Stuff so here was where I had a new session and I said create an asky of a cat please and then you can see here that um after some time it was able to generate this image as key art of a cat now there are some really cool
Applications that I do want to show you later on in the video which are really really cool and I’m going to show you those right now so essentially as this continued um and that was the just code there and of course that’s the python and then of course there was also this
Where plot a chart and this is basically um just this is basically the default one that they do give you it says plot a chart of Nvidia and Tesla stock price here today save the result of a file named Nvidia Tesla PNG so right here you can see the agent messages four messages
In 31 seconds and you can see Nvidia and Tesla PNG and then of course the corresponding code so you can see exactly we do have the image and this is of the real stock prices so all of this stuff is actually really really accurate and of course it shows you exactly how
It managed to do that so you can see the agent messages right there so it’s figuring out exactly how to do that using certain libraries using this Library um and yeah it’s really really cool so what was actually cool about this was that these agents were interacting with each other and you can
See that um one of them was actually saying that um there is a lack of certain libraries can you actually load them and then the second one performs it and then sends a report on the work so that’s why this is a pretty fascinating stuff so I think agents in the future
Are definitely going to be pretty crazy once we do get a more cohesive you know software that isn’t so not really technically Advanced but like just not as easy to use you can see that there’s a primary assistant and then the user proxy here a different agents uh working
Together and of course here’s where I asked for a Candlestick chart for Bitcoin and ethereum stocks and then here we can see that it is going ahead to complete that then after switching back to tabs we can see once again that this is completed now there was one
Really cool thing that um I was able to do with this that I think you guys are going to find uh really really fascinating so I asked it um you know with DAR 3 to generate create a picture of a whale in space and of course you
Can see it manages to do that pretty quickly so picture of a whale in space but that’s not the crazy thing okay so with these agents what you’re able to do is you’re able to generate like long pieces of content that’s really cool for example here I asked it to create a PDF
File with a presentation of the history history history for example here I asked it to create a history of London so create a PDF file with the history of London so essentially here what I’ve done is I’ve said create a PDF file with a you know presentation of the history
Of London so you about to see exactly what it was able to do so essentially right here it was able to give me uh three files you can see in 1 minute and 21 seconds and of course we can see this is a PDF file with the history of London
Then I said that I wanted more information cuz that’s not really that much and then I also said that I want some pictures too so I’ve asked to get some additional pictures of historical places in London and then of course you’re going to see that in around a
Minute or so seconds I was able to get you know all these kind of pictures of course they’re generated with dly 3 it’s not just going to like copy pictures from the internet but um you can see right here that this was really really fascinating especially the PDF 2 so here
We’re getting some historical images I’m guessing that’s of Big Ben in London and of course you can see the PDF is where I ask I need all this in a PDF file presentation and then you’re about to see here that all of this is able to be
Generated in a PDF file presentation so you can see the Tower of London then you can see Buckingham Palace then you can see West Minister Abby so yeah this is really really cool stuff and you can see that all of this is of course really real so there’s many different things
And many different um ways that you can interact with this agent multiple agent software I think it’s really cool and I think it’s a step in the right direction but in terms of what is coming next they do actually have a FAQ on what’s actually coming next so in the future
They detail that they’re going to have complex agent workflows so we’re working on integrating support for more sophisticated workflows such as group chat which is allow richer integration between multiple agents or dynamic topologies which is essentially you know when you have multiple agents in the group chat you’re going to be having
Multiple agents conversing on how to get a task done better also improved user experience so includes features like streaming intermediate model output for real-time feedback better summarization of agent responses information cost of each interaction um expansion of agent skills work so they’re going to add multiple different skills so we don’t
Know what skills is going to be and of course Community features so there’s going to be options to share sessions and more easily among users and contributing to a shared repository of skills agents and agent workflow testing 1212
Video “Microsoft’s Autogen 2 – Create Custom AI Agents” was uploaded on 01/14/2024 to Youtube Channel TheAIGRID