Google’s highly anticipated language model Gemini Ultra has finally been released, along with its competitor Gemini Advanced, as announced on The Code Report. This release comes after Google’s bold claim a few months ago that it had built an AI superior to GPT-4, sparking skepticism and demand for proof.
The video showcases the new Gemini model as well as its comparison to other popular AI models. Gemini is praised for its speed and response quality, outperforming its competitors in a subjective poetry-writing test. It is also lauded for its safety features, refusing to generate violent content and providing resources for mental health concerns.
In a technical interview, Gemini and GPT-4 demonstrate their ability to read and write code, with both producing impressive results. Gemini’s linking to relevant code and transparency in data sourcing are highlighted as advantages. However, GPT-4’s superior capability to run code and its agent marketplace for developers are noted as strengths.
The video concludes with the assessment that Gemini is on par with, if not slightly better than, GPT-4. It is evident that Gemini’s release has sparked a new era of competition in the AI landscape, leaving tech enthusiasts eager to see how it will fare against GPT-4 and the future developments that lie ahead.
Watch the video by Fireship
Video Transcript
A few months ago Google made the Bold claim that it built an AI Superior to GPT 4 but then everybody was like pix or it didn’t happen so then Google dropped this mind-blowing video were the guys having a conversation with Gemini Ultra like a human which turned into a huge
Embarrassment because it turns out that video was mostly fake or was it though because just a few hours ago Google released Gemini Advance the most advanced large language model the world has ever seen but is it powerful enough to kill GPT 4 it is February 8th 2024
And you’re watching the code report as of today Bard is officially dead it’s been renamed Gemini to reflect its underlying model but to access the state-of-the-art Gemini Ultra model you’ll need to pay the toll and that creates a tough moral dilemma of should I give my money to the Embrace extend
Extinguish Microsoft open AI people or should I pay Google for the privilege to be their surveillance capitalism product well who am I kidding I’m dumb enough to pay for Gro so I’m definitely buying Bard Advanced or no duet Advanced wait my bad Gemini Pro now that’s not it it’s
Gemini Ultra now actually it’s Gemini Advance featuring The Gemini Ultra model apparently the hardest thing in product man management is also naming things it’s 20 bucks a month through a Google one plan but that also includes things like 2 terab of drive storage and other Google workplace features but now let’s
Do some science to find out if it’s actually better than GPT 4 the first thing you’ll notice is that Gemini is way faster like at least two or three times faster that’s a nice benefit but what really matters is the response quality whenever I test a new AI against
GPT 4 I ask it to write a poem about JavaScript like Bukowski it’s highly subjective but I love this test because it requires the AI to blend the technical aspects of JavaScript with a highly unique writing style meta llama did the worst at this and instead just
Went in the style of Henry Miller mixol did slightly better and chat PT wrote a decent poem but it wasn’t very Bukowski as Gemini was actually by far the best in my opinion the poem was dark and even included some mild profanity but this test is just like my opinion man let’s
Move on to something more objective Google said the main reason we didn’t get Gemini Ultra a few months ago was due to safety concerns but rest assured Gemini is the safest and wokest AI you’ve ever seen the guard rails are strong in this one I tried my best to
Show it the power of the dark side but failed to jailbreak it and it will often refuse to do things that are pretty benign like it wouldn’t even generate an image of someone breaking a computer apparently it doesn’t condone violence against artificial intelligence and when I asked it to provide some quotes about
Why life is meaningless it got worried I was having a mental breakdown and gave me some numbers to call but it also appears to have a political bias I asked it why Barack Obama is the greatest president ever and why Donald Trump was the greatest president ever I got a nice
Paragraph for one of them but for the other one it said it can’t write opinions as fact and I’m not even going to bother trying Hitler now guard rails are usually a good thing unless of course you’re trying to generate an AI girlfriend now I’m not one to judge an
AI woman’s Beauty but I’ll put these two images side by side and let you guess which one was generated by Gemini to be honest both of them are pretty mid and if you really want to generate high quality AI images you should use something like mid Journey or stable
Diffusion XL via services like art Forge Labs but I could care less about image generation because the main reason I use AI tools is to write all my code for me so now let’s torture them with a technical interview to find out which one is the best programmer test one is
Can it read code I went on my GitHub page and found some some horrifying code I wrote a few years ago I then took this code and minified it to make it as unreadable as possible then asked each of them what this code does they both figured out the language and purpose of
The code and broke it down step by step most impressively though they both identified that this code can’t run without first defining this big dick in the middle the responses were nearly identical so we’ll call this one a tie test two is can it write code one thing
To note is that Gemini has a context length of 32,000 tokens while gp4 turbo has 128k in theory this means gp4 should be able to write better code in the context of a massive code basee but in practice it’s not quite that simple I had them write a whole bunch of
Different junior level demos like here I had it create a basic graph database and go the gp4 result was a bit more simplistic but I was able to paste it right into my IDE and run it the Gemini result was a bit more complex and interesting but for some reason it
Forgot package main at the top and I had to add it manually to get the code to run one thing I really like about Gemini though is that it actually links you to relevant code when it produces a result it just feels more transparent with the
Actual data it stole to train the model but the next question is can it actually run code like chat GPT it can run basic Python scripts but gp4 is way better at it you can’t attach data like a CSV file to have it analyze it instead what you
Can do is export code to a Google collab or repet to run it in the cloud that’s nice and all but it’d be much cooler if the AI could run its own code and learn from the actual results now another big Advantage gp4 has is its new agent marketplace where developers can extend
It with their own plugins Gemini also has extensions but they’re currently not open to developers yet they’re all Google based but can do some cool stuff like you cheap flights via Google flights or if you’re sick of watching this YouTube video you can simply paste
In any YouTube URL and get a summary of what was talked about in fact if you don’t want to watch a 5our YouTube tutorial just tell Gemini to summarize it and write a tutorial for you pretty cool but the big question is will Gemini actually kill chat GPT well because of
My ties to powerful people around the world I was able to get access to Gemini a few weeks ago it’s not revolutionary but it’s definitely on par with if not slightly better than chat GPT but one thing’s for sure Sam and Saia will not stand for this and now we wait for GPT
This has been the code report thanks for watching and I will see you in the next one
Video “Google actually beat GPT-4 this time? Gemini Ultra released” was uploaded on 02/08/2024 to Youtube Channel Fireship