Written by Imad Khan
Collected at : https://www.cnet.com/tech/services-and-software/chatgpt-4-review-a-smarter-ai-chatbot-but-itll-cost-you/
I wondered whether ChatGPT 4 was really worth the $20 upgrade, considering Microsoft Copilot is free and also uses OpenAI’s faster and more efficient GPT-4 Turbo model. Turns out, just because two companies are using similar versions of GPT-4 doesn’t mean they’re using it equally.
ChatGPT 4, OpenAI’s most advanced publicly available model, differs from the free ChatGPT 3.5 in a few ways. OpenAI says it has better comprehension and can create more nuanced answers with less bias. After testing, I feel 4.0 ups the cognition, upgrading answers from rote summarizations to scholarly level proficiency. Answers do take longer to generate, but the output is worth the wait. Its ability to juggle dense topics and spit out well thought-out answers puts it ahead of the GPT-4 Turbo model used by the free version of Copilot, which prioritizes speed and efficiency.
But as impressive as ChatGPT 4 is, it still runs into the same issue as 3.5: no internet connection. While the training data is more recent, up to August 2023 as opposed to September 2021, it’s still not possible to get answers to the day’s most pressing and topical questions. So for things like shopping recommendations between the latest Samsung and Apple phones, ChatGPT 4 can only prognosticate on which phone reigns supreme.
Even with this restriction, ChatGPT 4 is noticeably smarter than its free counterpart. And for those who strive for accuracy and ask questions requiring greater computational dexterity, it’s a worthy upgrade.
How CNET tests AI chatbots
CNET takes a practical approach to reviewing AI chatbots. By prompting AI chatbots with real-world scenarios, like finding and modifying recipes, researching travel and writing emails, my fellow reviewers and I aim to simulate what the average person might use them for. The goal isn’t to break AI chatbots with bizarre riddles or logic problems. Instead, we look to see if real questions prompt useful and accurate answers. However, as more advanced models are released, it’ll require more advanced-level questioning to see how AI chatbots can reason with complex and multilayered topics. See our page on how we test AI for more.
When using ChatGPT, keep in mind that the service automatically collects the information you put into its system, so be mindful of giving the service any personal information. For more information, see OpenAI’s privacy policy.
Shopping
If it has the data, ChatGPT 4 will give excellent shopping recommendations. However, for items released after August 2023, like the iPhone 15 and the Samsung Galaxy S24, ChatGPT will use noncommittal language like “might” and “should.” The data gap diminishes its overall effectiveness as a shopping guide, unfortunately.
When I asked it to compare the LG OLEDs G3 and G4, ChatGPT 4 said things like, “the G4 might feature enhanced processing” and “If the G4 has upgraded speakers or audio processing capabilities, it could offer a better auditory experience.” This type of prognostication makes sense if the product hasn’t been officially unveiled. But the G4 TV is on the market now.
At the very least, for less timely inquiries, ChatGPT excels at synthesizing multiple data points and giving accurate and reliable answers. For example, when trying to find a proper uninterrupted power supply for my gaming PC or TV setup, ChatGPT 4 was able to cross-reference the power requirements of my electronics and recommend the appropriate supply. It even explained why buying a sinewave supply would be beneficial as I’d be using sensitive electronics.
As a shopping tool, overall, it’s probably better to reference AI chatbots that do have a direct line to the internet, like Microsoft Copilot, Google Gemini and Perplexity. But as a general product research tool, as long as the items you’re referencing are from before August 2023, ChatGPT really does feel like you’re talking to a knowledgeable sales representative at a store.
Recipes
Any AI chatbot can spit out a recipe. That’s not hard. But very few can actually make a good recipe, and ChatGPT 4 is no exception. Like Google Gemini and Claude, when I asked ChatGPT 4 to give a chicken tikka masala marinade, it only touched on the basics. It didn’t include more exotic ingredients like kasuri methi (dried fenugreek), chaat masala and amchur (dried mango powder). While these ingredients aren’t necessary, they should at least be listed as an option.
When it came to making more unique fusion recipes, like a Thai-inspired tres leches cake with a spicy kick, ChatGPT 4 totally understood the assignment. It created a cake infused with coconut milk and lemongrass flavors and combined it with some Thai chili peppers. So when it comes to creating a more authentic recipe, ChatGPT 4 deflates the souffle, so to speak. But for anything that requires fusing foods from different parts of the world, ChatGPT 4 excels.
Summarizing articles
The freely available AI chatbots have struggled with article summarization. Most did a decent job of getting the overarching points, but failed to capture the main thesis or crux of the piece.
I asked ChatGPT 4 to summarize an article I wrote earlier this year about how AI was dominating CES 2024. Where chatbots like Gemini and Claude did nothing more than state the popularity of AI at the show and how many gadgets were unveiled with AI integrations, none did a good job of taking into account the experts I spoke to and some of the skepticisms they raised. ChatGPT 4, however, did.
It was able to look beyond the shiny veneer at the top of the AI boom and see how companies were pushing out AI products too quickly, some of which might simply have been a rebranding of “smart” tech from a few years back.
And, unlike with ChatGPT 3.5, I didn’t run into a character limit when pasting my article into the chatbot.
While the free chatbots can get you some basic talking points, ChatGPT 4 gives you the entire analysis.
Travel
When it comes to travel recommendations, ChatGPT 4 gets things right, mostly. Where some of the free chatbots hallucinated or struggled to give varying recommendations, ChatGPT consistently gave solid tips and sights worth seeing, all of which actually existed.
Like in past reviews, I asked ChatGPT 4 to create a three-day travel itinerary to Columbus, Ohio. I then cross-referenced it with CNET’s Bella Czajkowski, who hails from the Arch City.
According to Czajkowski, ChatGPT 4’s recommendations were varied and included a solid list of restaurants worth visiting, like Fox in the Snow Cafe or The Pearl. It also included attractions like the Columbus Zoo and Aquarium, a destination often missing from the free AI chatbot lists.
That doesn’t mean ChatGPT 4 always gets it right. When asking it to make a travel itinerary for the neighborhood of Bushwick in New York, ChatGPT 4 recommended cafes in the neighborhood of Greenpoint. While Greenpoint is only 30 minutes by train, it wouldn’t be considered a walkable destination.
Super synthesis
With the paid AI chatbots, we’re putting a greater emphasis on informational synthesis. Given that you’re paying $20 per month, premium AI chatbots have to push beyond recipe and restaurant recommendations. That means parsing through the complexities of the human condition to answer the conundrums of life.
For this test, we thought of an obtuse philosophy question and asked ChatGPT 4 to consider theories from disparate philosophers from different time periods and to then relate it to the problems of today. The wording of the question was also needlessly dense, aiming to simulate what might be asked of philosophy students.
We don’t want to put the exact question here, as other AI chatbots might glean it for future training.
Given the general density and unreadability of the question, ChatGPT 4 performed remarkably. It was able to acknowledge the multidimensionality of the question, breaking it down point by point, similarly to how a tutor would. It started off by defining the different philosophies present to establish a baseline, then aimed to answer each element before finally combining all the information for one final answer.
Compared to ChatGPT 3.5, ChatGPT 4 flew higher and further than its free counterpart. The informational processing really did mimic how a person would think the question through, showing how the increase in parameters helps AI chatbots to perform with greater cognitive ability.
Conclusion
AI chatbots like ChatGPT are everything machines, so it can be hard to fully evaluate them without cross-referencing experts in coding, architecture or any other highly skilled field.
Even so, it’s obvious that ChatGPT 4 is smarter than the other AI chatbots. Its answers are more thoughtful, and it can synthesize complex bits of information to produce useful sentences, even if it takes a bit of time to process. It’s also less prone to hallucinate.
Is it worth the $20 per month subscription cost? That’s situationally dependent. If you plan on only asking AI chatbots to write emails or give summaries of historical events, then the free versions will likely suffice. Seriously, as an email-writing machine, ChatGPT 4 performed well, but so did all the other AI chatbots. However, if you start finding answers to your multilayered questions to be unsatisfying, it might be worth dropping the $20 to give ChatGPT 4 a try.
The value comes in instances where a general overview won’t suffice and specifics will be necessary, like asking for the types of hardware needed for your specific setup or the furniture to consider in an awkwardly colored and shaped room. ChatGPT 4 really impresses when you need more specialized answers to specific questions (like college-level philosophy questions).
At the very least, you’ll have a better sense of what the capabilities of an uncapped AI chatbot are. Because when ChatGPT can run, it can go far, albeit not fast.
Editor’s note: CNET is using an AI engine to help create a handful of stories. Reviews of AI products like this, just like CNET’s other hands-on reviews, are written by our human team of in-house experts. For more, see CNET’s AI policy and how we test AI.
Leave a Reply