The AI Hype is OVER! Have LLMs Peaked?
The video titled “The AI Hype is OVER! Have LLMs Peaked?” debates the idea that generative AI, specifically Large Language Models (LLMs), may have reached a plateau in terms of capabilities and advancements. The narrator challenges this notion by discussing key examples and industry insights, including statements from industry leaders like Sam Altman and Mark Zuckerberg. They highlight the potential bottlenecks of energy and compute capacity that could slow down progress in the field of AI. However, the narrator also emphasizes the ongoing internal advancements within closed AI companies like OpenAI, indicating that breakthroughs are continually being made behind the scenes. They argue that the next 365 days will bring significant developments in AI, with the potential for multimodal agents and improved reasoning capabilities to revolutionize the field. Overall, the video encourages viewers to reconsider the notion of AI stagnation and to stay informed about the evolving landscape of artificial intelligence.
Watch the video by TheAIGRID
Video Transcript
so one of the current questions that I’ve been seeing floating around on Twitter and on social media is that AI hype might be wearing off and when I’m talking about AI hype this is referring to the trend that generative AI has I guess you could say reached an exhaustion point now I disagree and I’m going to tell you guys why but I’m going to go over some of the key examples and what people are stating so essentially there was a clip here and a lot of people were stating that this clip from Sam Altman people were stating that the hype is wearing off The Vibes are shifting you can feel it and basically in this clip clip here Sam Alman literally is he’s only really stating that you know I don’t care if we burn $50 billion a year we’re building AGI and it’s going to be worth it so it’s not a crazy crazy statement I think why people are stating that the hype is wearing off and that Vibes are wearing off and this video is going to be and essentially what this is referring to here is the fact that you know I guess a lot of people are thinking that you know I guess you could say the generative AI is currently slowing down in terms of the capabilities and what we’re likely to see in the future now like I said before it’s just mainly due to this clip and a bunch of other different factors but let’s get into how Cycles work in terms of technology and a graph that a lot of people have been referencing when they talk about llms plateauing so something that I also saw quite a bit being you know passed around as like this infographic is of course the Gartner hype cycle so essentially it’s just a graphical rep representation used to illustrate maturity adoption and social application of specific Technologies it basically just provides a conceptual framework that helps stakeholders and individuals understand how Technologies evolve from the time they introduced until they reach mainstream adoption and the hype cycle can be particularly useful for evaluating emerging Technologies like generative Ai and of course large language models so of course we essentially have the technology trigger this phase initially occurs when a new technology is conceptualized or when a significant development makes it publicly known creating interest and significant media attention for example this is when GPT 3.5 chat GPT was released and it demonstrated the ability to create text in the long format and then of course that’s when we got to GPT 4 which is stage two so this is where there is the peak of inflated expectations now I’ve got to be honest it’s not just llms that were going crazy at this point there was also 11 labs and other image generation services like mid journey and of course other services like stable diffusion so I would argue that the problem with this is that with generative AI experiencing some kind of increase in the actual media coverage I would say that this is something that is cumulatively increasing in terms of the expectations and that’s because like I said already there were many several different categories that came together at the same time and of course this could include llms being at the peak of inflated expectations now I want to say that I do disagree with this hype cycle for AI I do think that this is nowhere near it its peak where it should be but there of course is inflated expectations when a new technology comes to fruition a lot of people may exaggerate what the technology can really do for example some people say it’s going to replate replace entire careers or entire tasks and revolutionize entire Industries now whilst yes that might happen in the future I don’t think that that is completely happening with GPT 4 and gbt 3.5 so some are arguing that this is where we are at currently okay and then of course the trial of disillusionment and this is where Technologies enter this phase when they fail to meet the inflated expectations and stakeholders become increasingly delusion so issues related to the technology start to arise such as the biases and large language models need for vast data training the high operational costs the high inference costs the environmental impacts they become more apparent and then they become criticized and one of the main things that many people are talking about with llms is of course things like the hallucination and the high inference costs and of course the training cost because these models are certainly not cheap and it seems like there’s only a few companies that can run and train these large models now of course there we have the slope of Enlightenment and this is essentially where with l M all of these previous issues get ironed out so things like the issues of hallucination and the biases they get ironed out right here and this is where as more experimentations as more implementations occur the market matures and second third generation products appear and with llms this would involve developments that address the earlier criticisms such as improving the model efficiency the inference costs reducing the biases and of course the reliability of the model and this is arguably where people think AI is going to go technology becomes stable and accepted and this means widespread adoption however I think the graph isn’t going to look anything like that I think it’s probably going to look something like this where we go up we dip a little bit and then we continue a trajectory upwards because like I said before whilst yes that it does seem to many different pieces of Statistics that it looks like we are slowing down in terms of AI versus human performance and even on this Stanford AI index we can see that AI has surpassed human performance on several benchmarks including someon image classification visual reasoning and English understanding yet it Trails behind on more complex tasks like competition level mathematics visual Common Sense reasoning and planning and you can see here that if we take a look at actually how the AI is moving we can see that in image classification visual Common Sense reasoning natural language inference all of these seem to be coming towards the human level Baseline but they don’t seem to be going upwards you know like in a crazy level on on the graph and some people would argue that this is because AI generative AI large language models whatever you want to call it if you just want to group everything together that we have reached our limit in terms of where we are and new architectures are going to be needed now I would firmly disagree with this for several reasons that I’m about to explain and I’m going to show you guys some really key evidence on why things are about to actually get very very crazy in the world of AI and why we’re about to go into a very very abundant era due to AI so one of the things that many people are not actually taking into account is the fact that currently there are many different things going on in the world of AI that people aren’t paying attention to some people are just paying attention to large language models and this doesn’t make sense because there are vast and many different categories in which AI is currently exceeding for example in voice recognition and in voice generation opening ey has recently developed their voice engine which was actually something that they developed in around 2022 and it was basically a state-of-the-art system that could recreate anyone’s voice in addition to that if you do remember open ey also did talk about and quote unquote release their sore up model to Showcase us how far they’ve come in creating a text video model now I think you have to understand how crazy this is because for people to say that generative AI is stagnating is a crazy statement when literally this year we literally got Sora which blew everyone’s Minds in terms of the capabilities so with Sora I still find it absolutely incredible that people can say AI is stagnating because with Sora as you’ve all seen this was something that was truly just mind-blowing this piece of technology showed us how crazy it is when you get a group of AI researchers dedicated to doing something and I’m going to give you guys a quick memory joke remember that opening ey isn’t a company that’s Focus fed on video generation this is just a subsection of their company so the video generation aspect was something that I guess they just wanted to see if they could do well at and they literally surpassed state-of-the-art models Google Runway pabs they completely surpassed them and this is absolutely crazy I mean the demo literally absolutely shocked everyone I was truly truly speechless when this technology was here and I’m someone that pays attention to all of the AI news and of course we do have Devon cognition Labs released Devon their first AI software engineer and this was a AI rapper around GPT 4 but it did a few things in a unique way where it was able to surpass GPT 4 on certain software engineering benchmarks so one of the first things the point here I’m making is that you might think that llms are stagnating but that is not the truth at all we’re going to get into llms later but overall voice engine Sora and Devon show us that generative AI is really really not going to be stagnating anytime soon but if that didn’t convince you let me show you guys some of the recent statements that show you that we’re absolutely in for a crazy ride so Sam Alman recently said in an interview at Stanford’s entrepreneurship talk he spoke about gp4 now remember GPT 4 currently is a state-of-the-art system meaning that it is the best of the best that we can currently get our hands on for public use which means that Sam Alman currently probably has access to Frontier that are being developed by open Ai and remember the big Labs like anthropic and Google are currently behind them in terms of what they’re creating so take a listen to this statement open ey is phenomenal chat gbt is phenomenal um everything else all the other models are phenomenal it burned you’ve burned $520 million of cash last year that doesn’t concern you in terms of thinking about the economic model of how do you actually where’s going to be the monetization source well first of all that’s nice of you to say but Chachi BT is not phenomenal like Chachi BT is mildly embarrassing in a best um gp4 is the dumbest model any of you will ever ever have to use again by a lot um but you know it’s like important to ship early and often so if you weren’t paying attention there Sam mman literally just said that this is going to be the dumbest model that we will have to use or that we will have had to use by far so he didn’t just state that this was going to be an incremental increase he clearly stated that gp4 was dumb he stated this model was not you know great he stated that this model was not that good he clearly stated that GPT 4 a current state-ofthe-art system that people were literally able to get you know increasingly capabilities just by wrapping the system and being able to do software engineering tasks and a lot of people even using GPT 4 to be able to train robotic systems like recently we saw in a research paper and this was literally where we had language model guided Sim to real transfer so we basically had large language models were basically writing the reward functions for an AI system and it was able to do it very effectively and it literally just went from simulation to real life which means that this is going to immediately speed up how quickly we’re able to train robots and have them you know doing well in novel environments it’s pretty pretty insane so it’s an llm guided Sim toal approach so that is pretty crazy and remember I’m guessing that they were using GPT 4 that’s actually a study that I haven’t covered just yet but the point is is that samman states that the current state-of-the-art model the current model that other industry labs are trying to be millions and millions of dollars he states that that model is bad and it’s dumb he didn’t just say it was kind of smart he said that it was dumb which leads me to believe that we are truly truly not even scratching the surface for what AI systems are he could have just said future models will be interesting he could have just said they will be you know kind of good but he literally said okay I’m going to play it one more time first of all that’s nice of you to say but Chachi BT is not phenomenal like Chachi PT is mildly embarrassing at best um GPT 4 is the dumbest model any of you will ever ever have to use again Again by a lot um but you know it’s like important to ship early and often so that just goes to show how clear there’s going to be a distinction in the future now he also stated that there will be a massive jump from DPT 3.5 to 4 and there will be a similar jump from GPT 4 to GPT 5 so it’s important to know the jump from GPT 3.5 to GPT 4 was incredible because of GPT 3.5 limitations it meant that it couldn’t be extended to certain tasks but if we have that same jump from GPT 4 to GPT 5 then things are truly about to change there’s so many questions uh first of all also amazing it’s looking back it’ll probably be this kind of historic pivotal moment with 35 and four which had your BT maybe five will be the pivotal moment I don’t know hard to say that looking forwards we never know that’s the annoying thing about the future it’s hard to predict but for me looking back GPT 4 Chad GPT is pretty damn impressive like historically impressive so allow me uh to ask what’s been the most impressive capabilities of GPT 4 to you and gp4 turbo I think it kind of sucks H typical human also gotten used to an awesome thing no I think it is an amazing thing um but relative to where we need to get to and where I believe we will get to uh you know at the time of like gpt3 people were like oh this is amazing this is this like Marvel of technology and it is it was uh you know now we have gp4 and look at GB3 and you’re like that’s unimaginably horrible um I expect that the Delta between 5 and four will be the same as between four and three and I think it is our job to live a few years in the future and remember that the tools we have now are going to kind of suck looking backwards at them so you can clearly see here that he’s basically stating that a year from now when we have gbt 5 or the next level of Frontier Model is released we’re going to look back at GPT 4 and think it’s pretty pretty bad so as much as people are stating that AI hype is wearing off Sam alman’s feeling it there’s this Gartner hype cycle it’s important to remember the subtle cues the subtle tells the subtle statements that we’ve clearly seen from industry leaders about these future future models that only they currently have access to now if generative AI was actually slowing down it would be due to a bottleneck and there are only a couple bottlenecks that are really the issues okay one of the issues is of course energy and this one isn’t really talked about enough because energy isn’t something that people think of but trust me when I say these inference costs are really really expensive and a lot of people you know including Mark Zuckerberg are basically stating that one of the major limitations for future AI systems and where things might actually start to slow down is due to the energy cost the rampant energy cost for these AI systems and just how much electricity they consume and there were even a few rumors talking about a GPT 6 training cluster project that would arguably shut down the power grid something along those lines I know it does sound crazy and I know rumors are rumors but energy is expensive and a lot of it is required to run inference on these large language models which is why they often restrict us I think there was this issue of um GPU production yeah right so even companies that had the money to pay for the gpus um couldn’t necessarily get as many as they wanted because there was there were all these Supply constraints now I think that’s sort of getting less so now I think you’re seeing a bunch of companies think about wow we should just like really invest a lot of money in building out these things and I think that will go for um for some period of time there is a capital question of like okay at what point does it stop being worth it to put the capital in but I actually think before we hit that you’re going to run into energy constraints right because I just I mean I don’t think anyone’s built a gigawatt single training cluster yet I me just to I guess put this in perspective I think a gigawatt it’s like around the size of like a meaningful nuclear power plant only going towards training a model and then you run into these things that just end up being slower in the world like getting energy permitted is like a very heavily regulated government function and if you’re talking about building large new power plants or large build outs and then building transmission lines that cross other private or public land that is just a heavily regulated thing so you’re talking about many years of lead time so if we wanted to stand up just some like massive facility um to power that I I think that that is that’s that’s a very long-term project so basically what Mark Zuckerberg is stating here is that energy is going to be a huge bottleneck because unlike software where you can make it more efficient and you can do things quickly or you can get a GPU produced quickly trying to build a nuclear power plant it takes time trying to build this these kind of infrastructure it’s not something you could just do in a day and it’s something that’s quite important when it comes to ative AI because as the race continues they’re going to be spending a lot more money they’re going to be spending a lot more in terms of how much they’re spending on cooling because these gpus heat up quite a bit so I think this is something that you know this is probably where the constraints actually will come from in the future and this is where things might actually slow down a little bit because there are going to be a few things that make this truly hard to continue with I think we would probably build out bigger clusters than we currently can if we could get the energy to do it so I think that that’s um that’s fundamentally money bottl in the limit like if you had a trillion dollar I think it’s time right um but it depends on how far the the exponential curves go right like I think a number of companies are working on you know right now I think you know like a lot of data centers are on the order of 50 megawatts or 100 megawatts or like a big one might be50 megawatts okay so you take a whole Data Center and you fill it up with just all the stuff that you need to do for training and you build the biggest cluster you can I think you’re that’s kind of I think a bunch of companies are running at stuff like that so it will be interesting to see where gen of AI does fall down but like I’ve said it’s not slowing down anytime soon and if you remember open a and Microsoft are building a100 billion Stargate AI supercomputer to power the AGI or ASI now there’s also another bottleneck which is what I’ve titled here which is the compute problem essentially this just means that the compute capacity for AI systems is far too great and it kind of exceeds the demand that we require so that’s why they’re building out this1 billion supercomputer to meet the demands of future generative AI systems or to essentially power the next Industrial Revolution because my oh my if an AGI or air size here it’s going to be used pretty much everywhere to power the economy and you’re going to need the compute and the infrastructure to do that and currently this is one of our biggest things because openi and Microsoft don’t even have enough chips and don’t even have enough computes currently to compete with the likes of Google so what we have here is we have updates to the chips and you can see that nvidia’s recent Blackwell is pretty pretty incredible and this was one of the most important developments for AI because this accelerates the training of large language models and generative AI systems so the Blackwell GPU architecture with its 208 billion transistors and its enhanced Transformer engine is designed to dramatically increase increase the training of large language models like GPT 4 according to Nvidia Blackwell can provide up to 30 times higher performance for generative AI inference compared to the previous h100 gpus four times faster training performance for large language models and this essentially means large language models like GPT 4 which took around 90 days to train on 8, h100 gpus consuming 15 megawatt of power could potentially be just trained in just 30 days on two 2,000 black C gpus only using 4 megawatt which is pretty pretty incredible and that curve definitely reminds me of some that we’ve seen before where things are starting to exponentially increase so the point right here is that these are the actual bottlenecks of generative AI because a lot of people are thinking that you know things are slowing down things are getting worse and trust me guys if you been paying attention things I would argue are actually speeding up internally and one of the things that I didn’t even include in this presentation because I forgot about it is the fact that right now it’s like open ey lit a match under every other company because now other companies are realizing that whoa there’s a huge huge AI race going on and if we partake we could definitely be getting billions and billions of dollars and that means that other companies and other startups are all rushing down the corridor to see if they can get piece of the piie which means that we’re about to see a complete Revolution and a complete new industry in terms of all of these products and services now one of the biggest things that I think that most people need to consider is that open AI are no longer open okay and if there’s one thing you take away from this video please understand this okay things might be slowing down externally but things are not slowing down internally and what I mean by that statement is that currently we’re at a stage where things have moved from an open research environment to a closed research environment the reason this has happened is because opening up they’re no longer essentially a company that’s just focused on Research they are a business and businesses you know they hide their secrets and they hide their Innovations because they don’t want their competitors to have them if open I shared all their secrets then other companies could easily build gbt 4 with remarkable accuracy and op essentially has secret Source the thing is openai also doesn’t publicize their research I’m sure breakthroughs are made every single month okay and you have to think about it if openai did their whatever they did with gbt so long ago they must have some secret kind of breakthrough they must have some secret source and they must have something that others don’t which essentially means that open ey are making consistent breakthroughs and remember Sora they had Sora we had no news no indication that they were even developing some video AI there was literally no indication whatsoever there was no interview from Sam Alman there was literally nothing we could have picked up on on the fact that they were even training such a model and boom they just you know put it out into the open the point is is that we know we have no idea on what’s going on at you know closed AI open AI whatever you want to call it the point is is that internally I can guarantee you guys they are like 2 to 3 years ahead from where they are and the point there is that whilst you might think ah they haven’t released anything in a while that doesn’t mean things are slowing down it just means that they’re thinking okay how can we not Shock the public with this next release that we know is literally going to take everything remember other companies are still playing catchup gbt 4 finished training in August of 2022 which means that we are very very lucky because we’re going to be in for a real surprise when gbt 5 gets here now another thing to note as well okay is that GPT 4 being The Benchmark does not mean Plateau the problem with this and like I said before this is a business which mean things are going to change GPT 4 is currently the Benchmark which means that companies are incentivized to train their models to surpass GPT 4 and then release that model the reason this creates the illusion that things are plateauing around GPT 4 is because these companies are no longer incentivized to go duly pass GPT 4 they’re only incentivized to just beat it and that is because of course with GPT 4 that is something that people State oh it’s the best system is the best system so if another company like gemini or anthropic or Google can come out and say look our system surpasses gbt 4 or benchmarks they’re going to immediately release that model after it’s fine- tuned or after after it’s whatever they’ve done with it and then run with that so that they can Market that and get the customer base because they know that open AI are waiting to release GPT 5 potentially after the elections and that gives them some time to reclaim the market share understand that where GPT 4 is is just an indication of where other models are going to stop and if you think that that is just a pure speculative argument look at how close gp4 is to some of these models you have to understand that if they didn’t beat gp4 they wouldn’t be releasing these models this is 86.4% they literally got it up to 86.8% another one here 92% they got this up to 95% okay it’s not like it’s completely surpassing them and I guess some people like look it all you know slows down around here no they just want it to be as close as possible so that they can get this out as quickly as possible because they know by the time open hour releases next again they’re going to be even behind and you can see here that Gemini Ultra a lot of people were even debating this because this was Chain of Thought at 32 um and that what they did in order to beat this because I’m guessing that when they had finished training the model and when they finished fine-tuning it they had to you know increasingly developed certain methods just to get this metric right here and that’s why I state that gp4 being the Benchmark does not mean we’re currently at a plateau at all because it’s likely that these companies are just benchmarking their models up to GPT 4 so that they can get them out now here’s why things are going to go even crazy and remember I said this because the next 365 days are going to be absolutely insane agents are still early okay and someone actually recently created a benchmark where they’re talking about multimodal agents for open-ended tasks in real computer environments and essentially with this you can see that humans can accomplish over 72% of the task and the best current AI agent can only do 12.24% what happens when AI agents can get to above 80% that is truly going to change everything and with the advanced reasoning and with the advanced capabilities of future models we going to see a future that we’ve never seen before now there was also something that I covered in a previous video that I’m guessing the majority of people are just completely glossing over and I’m sure it’s because during the video I was kind of sick because I had some kind of flu whatever but I still made the video anyways so essentially there was this thing right here okay this is mesa’s kpu now this is a little bit speculative because they haven’t released too much information but if you check the benchmarks here you can see that this surpasses claw 3 Opus Gemini Ultra and Mr large at all benchmarks okay and this is because they use a advanced reasoning engine on top of gp4 Turbo now this is pretty interesting because this shows us that we are still very early on the reasoning capabilities which is why I argue that samman here says that gp4 is dumb and why he also says here that it was not a very good AI system so there was one demo released by M’s kpu in which they showcased an AI system actually doing reasoning with an advanced task and recently on their Twitter I’m not sure why it’s not getting any love or any actual you know tweets about it they’ve shown that this system is able to you know use some reasoning steps and this is their system they’re messing around with it and they’re showing that it’s able to complete a lot of tasks really really correctly so you have to remember that internally things are going at light speed things like qar things like other companies now trying to get a piece of the pie uh incorporating different reasoning engines on top of gbt 4 are going to push things further and remember it was recently that Andrew NG actually spoke about agentic workflows and basically said that GPT 3.5 zero shot was 48.3 five and of course it was Andrew NG that did some research and found out that GPT 3.5 zero shot was 48.1% correct GPT 4 zero shot does better at 67% but the Improvement was dwarfed by incorporating an iterative agent workflow and wrapped in an agent Loop GPT 3.5 gets up to 95.1% so the point here is that there’s still a lot of different architectures that we haven’t fully explored with some of the AI systems that we do have which means that we are far far far away from any sort of plateau and things are going to keep increasing number one we’ve got the gpus increasing in terms of efficiency we’ve got the data centers we’ve got all of these things getting increasingly better and of course we’ve got the fact that internally open AI they are blisteringly so far ahead that I’m guessing that things are going to be shocking when they are finally released
Video “The AI Hype is OVER! Have LLMs Peaked?” was uploaded on 05/09/2024 to Youtube Channel TheAIGRID