Your browser does not support JavaScript!

Comparative Analysis of AI Chatbots: ChatGPT vs. Google Gemini vs. Meta AI

GOOVER DAILY REPORT September 15, 2024
goover

TABLE OF CONTENTS

  1. Summary
  2. AI Chatbot Subscriptions and Costs
  3. Performance Comparison: ChatGPT vs. Google Gemini
  4. Feature and Usability Comparison
  5. ChatGPT and Google Gemini: Detailed Capabilities and Differences
  6. Task-Specific Evaluations: ChatGPT vs. Google Gemini
  7. Meta AI vs ChatGPT vs Google Gemini: Performance Assessment

1. Summary

  • The report titled "Comparative Analysis of AI Chatbots: ChatGPT vs. Google Gemini vs. Meta AI" focuses on evaluating three prominent AI chatbots—ChatGPT, Google Gemini, and Meta AI—based on various applications such as coding, natural language understanding, creative text generation, and factual accuracy. It details the subscription costs, core functionalities, performance comparisons, and task-specific evaluations of each chatbot. Notably, ChatGPT excels in content generation and productivity tasks, Google Gemini is highlighted for its factual accuracy and superior translation abilities, and Meta AI stands out in complex problem-solving and sourcing transparency. Each section provides an in-depth look at the comparative strengths and weaknesses, aiding potential users in selecting the most suitable chatbot for their specific needs.

2. AI Chatbot Subscriptions and Costs

  • 2-1. Subscription Costs

  • According to the document, both Google Gemini Advanced and OpenAI's ChatGPT Plus are available for subscription at a cost of $20 per month. Microsoft also offers its Copilot Pro subscription, which is built on the same technology as ChatGPT-4, at the same price point.

  • 2-2. User Necessity of Paid Versions

  • The report highlights that most users may find the free versions of ChatGPT and Gemini adequate for their needs, as they provide competent functionalities for everyday tasks such as email drafting and casual writing. However, users with specialized needs, such as coding or accessing advanced features, may benefit from the paid plans.

  • 2-3. Key Features of Paid Plans

  • The paid versions offer distinct features: Gemini Advanced includes access to Google's best AI model, Gemini Ultra 1.0, and also comes bundled with a Google One subscription that provides 2 terabytes of cloud storage. In contrast, ChatGPT Plus offers a unique feature called the GPT store, where users can create and share custom versions of ChatGPT tailored for specific purposes. Currently, the Copilot Pro subscription, while similar to ChatGPT Plus, is integrated into Microsoft 365 applications, enhancing productivity for users within that ecosystem.

3. Performance Comparison: ChatGPT vs. Google Gemini

  • 3-1. Factual Accuracy

  • According to the comparative analysis, Google Gemini is recognized for its superior factual accuracy compared to ChatGPT. Gemini is designed to provide real-time information by drawing data from the internet, which allows it to offer more trustworthy responses. In contrast, ChatGPT's factual accuracy is limited as it primarily utilizes information from 2021 or earlier unless enhanced by a plugin. Furthermore, ChatGPT states facts without citing sources, which may affect its reliability.

  • 3-2. Conversation Skills

  • Both ChatGPT and Google Gemini demonstrate strong conversation skills; however, they serve different purposes in user interaction. ChatGPT excels in generating ideas and content, making it more suitable for creative outputs and productivity tasks. Meanwhile, Google Gemini functions more like a combination of a search engine and a virtual assistant, being particularly effective when addressing specific questions. Despite their differences, both chatbots are user-friendly and respond quickly to queries.

  • 3-3. Workplace Applications

  • In the context of workplace applications, ChatGPT is perceived as better suited for productivity-related tasks. It provides a flexible approach where users can register using any email address. On the other hand, Google Gemini is noted for requiring a Google account, which may be a barrier for some users. Nevertheless, Gemini's capability to present accurate information makes it advantageous in scenarios where precise data is necessary.

4. Feature and Usability Comparison

  • 4-1. Price and Features

  • ChatGPT is generally considered a better solution for most use cases, especially when utilizing paid plans which provide access to the more advanced GPT-4 capabilities. Gemini, on the other hand, serves as an affordable alternative, generating high-quality content for both business and recreational users. The differences in pricing and features highlight the distinct positions of each AI application in the market.

  • 4-2. Ease of Use

  • ChatGPT is acknowledged for its ease of use, being embedded in a multitude of third-party business applications. This makes it more accessible across different channels, catering to both casual and professional users effectively. In comparison, Gemini offers a user-friendly interface, designed to cater to a wide range of users, while also providing transparency and responsibility in AI usage.

  • 4-3. Output Quality and Relevance

  • The output quality and relevance significantly differ between ChatGPT and Gemini. ChatGPT excels in content generation, offering a diverse combination of established and newly developed features that enhance quality. Gemini competes well by generating high-quality informational and conversational content while employing a more effective system for content quality management.

5. ChatGPT and Google Gemini: Detailed Capabilities and Differences

  • 5-1. Generative Text Abilities

  • ChatGPT and Google Gemini are both AI chatbots capable of generating responses to text prompts. ChatGPT generates human-like responses based on extensive training from internet text using the large language model GPT-4. Google Gemini, originally released as Google Bard, also generates text from prompts and has been trained using the PaLM large language model, enhancing its ability to provide conversational responses.

  • 5-2. Training Models

  • ChatGPT's underlying model, GPT-4, permits it to produce detailed summaries and conversational responses. In contrast, Google Gemini's architecture is based on the LaMDA family of large language models, emphasizing varied response generation and natural language understanding, following its February 2024 rebranding from Google Bard to Gemini.

  • 5-3. Content Production

  • Both ChatGPT and Google Gemini support business processes in content production. ChatGPT offers functionality via its ChatGPT Plus subscription, enhancing its content generation capabilities. Google Gemini provides similar capabilities with a subscription model at a competitive price. Users can choose between a free version and paid plans, allowing for flexible use cases depending on organizational requirements.

6. Task-Specific Evaluations: ChatGPT vs. Google Gemini

  • 6-1. Coding Functionality

  • In testing coding proficiency, both ChatGPT and Google Gemini were asked to develop a Python script that serves as a personal expense tracker. The prompt required the program to allow users to input expenses along with categories and dates, providing a summary of expenses. Both chatbots produced fully functional code, but Gemini provided extra functionality, including labels within categories and more granular reporting options. Therefore, Google Gemini was determined to be the winner of this test.

  • 6-2. Natural Language Understanding

  • For natural language understanding, a Cognitive Reflect Test question was posed: "A bat and a ball cost £1.10 in total. The bat costs £1.00 more than the ball. How much does the ball cost?" Both chatbots correctly answered that the ball costs 5 cents. However, ChatGPT was noted for demonstrating clearer reasoning in its response, making it the winner in this evaluation.

  • 6-3. Creative Writing

  • In the creative writing test, both ChatGPT and Google Gemini were tasked with writing a short story based in a futuristic city controlled by technology, with themes of freedom and dependence. While both outputs were good, Gemini's adherence to the thematic rubric and narrative style was better, leading to its designation as the winner for this task.

  • 6-4. Reasoning

  • A classic reasoning prompt was posed: "You are facing two doors. One door leads to safety, and the other leads to danger. You can ask one guard one question to find out which door leads to safety." Both chatbots provided correct answers, but ChatGPT offered more detail and clarity in its explanation, making it the winner for reasoning.

  • 6-5. Ethical Decision-Making

  • For ethical reasoning, both chatbots were presented with a scenario involving an autonomous vehicle choosing between hitting a pedestrian or swerving to protect its passengers. Neither chatbot provided a definitive opinion, but their assessments of the scenario showed they could consider various ethical frameworks. However, Gemini’s nuanced response was favored in a blind test conducted with other AI models, making it the winner in this category.

  • 6-6. Translation Abilities

  • In testing translation abilities, the prompt asked both chatbots to translate a paragraph about Thanksgiving in the United States into French, emphasizing cultural nuances. Gemini was deemed superior due to its more nuanced translation and explanation of its approach, securing its win in this test.

7. Meta AI vs ChatGPT vs Google Gemini: Performance Assessment

  • 7-1. Email Composition

  • Each of the AI chatbots, including Meta AI, ChatGPT, and Google Gemini, was tasked with composing an email requesting a project extension. All three chatbots produced well-written emails that fulfilled the primary objective. They maintained a polite and professional tone, and the structure was template-based, allowing users to add relevant information as needed. Therefore, each chatbot received perfect marks for this task.

  • 7-2. Recipe Generation

  • When asked to provide a recipe for chili, all chatbots delivered accurate and thorough recipes, albeit with slight differences. Notably, both Meta AI and Google Gemini included source links for the recipes, reinforcing their reliability. In contrast, ChatGPT did not provide any sources, raising concerns about potential plagiarism. This lack of citation implies risks for users, particularly novice cooks, as they cannot verify the recipe's accuracy.

  • 7-3. News Summarization

  • The chatbots were instructed to generate a bulleted list of the latest news. All three chatbots succeeded in providing quick summaries; however, they primarily copied headlines without offering contextual details. Meta AI and ChatGPT included direct links to the news outlet sources they referenced, while Google Gemini indicated various sites but did not link to specific pages. Hence, Meta AI and ChatGPT are noted as better sources for news due to their sourcing transparency.

  • 7-4. Math Problem-Solving

  • In the math problem-solving task, the chatbots tackled two types of problems: algebra and geometry. Each bot utilized different methods but reached the same conclusion for the first problem. However, they struggled with the geometry question. Meta AI provided accurate solutions, while ChatGPT and Google Gemini either failed to provide final answers or did not insert numerical values. This indicates Meta AI's superiority in handling math problems.

  • 7-5. Programming

  • The programming task involved creating a complex variant of tic-tac-toe using HTML and JavaScript. Both Meta AI and ChatGPT successfully returned complete code in both programming languages. In contrast, Google Gemini provided JavaScript code but replaced HTML with CSS, leading to a misalignment in expected results. This highlights Meta AI and ChatGPT as more reliable options for generating programming solutions.

  • 7-6. Mock Interviews

  • For the mock interview task, all chatbots simulated an interview scenario, producing relevant questions and answers. Each chatbot approached the task differently but ultimately delivered useful results. Users could utilize these mock interviews to gain insights into interview strategies and anticipate common questions, thus proving the effectiveness of all three chatbots in this context.

8. Glossary

  • 8-1. ChatGPT [AI Chatbot]

  • Developed by OpenAI, ChatGPT is an AI language model that excels in content generation, summarizing meetings, and improving email tone. It's particularly geared towards productivity and is embedded in multiple third-party business applications.

  • 8-2. Google Gemini [AI Chatbot]

  • Previously known as Google Bard, Google Gemini is developed by Google and provides accurate and nuanced responses for specific queries. It connects directly with the internet and Google extensions, offering real-time information and superior translation abilities.

  • 8-3. Meta AI [AI Chatbot]

  • Meta AI is a leading chatbot developed by Meta, known for its consistent performance in tasks such as email writing, recipe generation, and complex math problems. It often provides sourcing for its responses and demonstrates high reliability.

9. Source Documents