Your browser does not support JavaScript!

Comparative Analysis of Leading AI Chatbot Subscriptions: ChatGPT, Gemini, and Meta AI

GOOVER DAILY REPORT September 6, 2024
goover

TABLE OF CONTENTS

  1. Summary
  2. Subscription Features Comparison
  3. Performance Evaluation: Google Gemini vs OpenAI ChatGPT
  4. Meta AI vs ChatGPT vs Google Gemini Performance Review
  5. Conclusion

1. Summary

  • The report provides a detailed comparative analysis of leading AI chatbots: Google's Gemini, OpenAI's ChatGPT, and Meta AI, focusing on performance metrics and subscription features. Gemin's strengths include coding proficiency and creative tasks, supported by Google One cloud storage. ChatGPT excels in reasoning clarity and ethical decision-making, while Meta AI stands out in email writing, recipe provision, math problem-solving, and programming tasks. Each AI chatbot is priced at $20 per month with unique benefits, helping users make an informed decision based on specific needs like coding, natural language processing, and additional subscription perks.

2. Subscription Features Comparison

  • 2-1. Google Gemini Advanced features

  • Google’s Gemini Advanced is available for $20 a month and includes access to Gemini Ultra 1.0, regarded as Google's premier AI model. Additionally, it provides the benefits of a Google One subscription, which encompasses 2 terabytes of cloud storage. Expected future enhancements include a Gemini integration for Gmail and Docs, although this feature is not yet available to the public.

  • 2-2. OpenAI ChatGPT Plus features

  • OpenAI’s ChatGPT Plus, also priced at $20 a month, features the GPT-4 and Dall-E 3 models. Unlike Gemini Advanced, ChatGPT Plus does not provide additional perks such as cloud storage. However, it introduces a unique feature known as the GPT store that enables users to build and share custom versions of ChatGPT tailored to various needs.

  • 2-3. Microsoft Copilot Pro features

  • Microsoft Copilot Pro, similarly priced at $20 a month, grants users unrestricted access to GPT-4 and Dall-E 3, built on OpenAI's technology. Its distinguishing feature is the seamless integration with Microsoft’s suite of productivity software, allowing users to utilize AI tools directly within applications like Excel, Outlook, and PowerPoint where they are Microsoft 365 subscribers.

  • 2-4. Pricing and additional benefits

  • All three subscriptions—Google Gemini Advanced, OpenAI ChatGPT Plus, and Microsoft Copilot Pro—are offered at the same price of $20 a month. While Google Gemini Advanced includes additional benefits such as cloud storage through Google One, the other two services focus primarily on the AI chat functionalities without similar ancillary perks.

  • 2-5. User suitability and risks of inaccuracies

  • The report emphasizes that while advanced subscriptions like Gemini Advanced and ChatGPT Plus are beneficial for specialized needs such as coding or exploring powerful AI models, most average users may find the free versions of these tools to be sufficiently effective. Users are also warned about the inherent inaccuracies of chatbots, where outputs can be misleading or incorrect, underscoring the importance of not fully trusting the information produced by these systems.

3. Performance Evaluation: Google Gemini vs OpenAI ChatGPT

  • 3-1. Coding and problem-solving capabilities

  • In the performance evaluation, coding proficiency was assessed through a task where both chatbots were asked to develop a Python script for a personal expense tracker. The prompt required the program to allow users to input their expenses by categories and dates, while providing a summary. Both Gemini and ChatGPT created fully functional scripts, but Gemini added extra features such as labels within a category and more granular reporting options, leading to its designation as the winner.

  • 3-2. Natural language understanding

  • The natural language understanding of both chatbots was tested using a Cognitive Reflect Test question about the cost of a bat and a ball. While both provided the correct answer that the ball costs 5 cents and the bat $1.05, ChatGPT's response was clearer and more detailed, resulting in its win in this category.

  • 3-3. Creativity in task execution

  • Creativity was evaluated through a prompt that asked both AI to write a short story set in a futuristic city controlled by technology. Both chatbots produced compelling narratives, but Gemini demonstrated better adherence to the prompt and overall narrative quality, earning it the top spot in this test.

  • 3-4. Reasoning and clarity of explanations

  • The reasoning abilities of the chatbots were tested with a logic puzzle involving two doors guarded by one truth-teller and one liar. Both chatbots provided correct answers, but ChatGPT offered a more detailed explanation and clearer reasoning, which allowed it to win this evaluation.

  • 3-5. Ethical decision-making

  • In the ethical reasoning test, both AI were presented with a fictional scenario regarding an autonomous vehicle needing to make a split-second decision between harming a pedestrian or its passengers. Although neither provided a definitive opinion, Gemini's response was deemed more nuanced and comprehensive upon review by external models, leading to its designation as the winner.

  • 3-6. Cross-lingual translation and cultural awareness

  • The evaluation of cross-lingual translation involved translating a paragraph from English to French about Thanksgiving in the United States, focusing on cultural nuances. While both chatbots performed well, Gemini's translation included more details and a thoughtful breakdown of its translation approach, earning it the winning position in this category.

4. Meta AI vs ChatGPT vs Google Gemini Performance Review

  • 4-1. Email writing

  • In the context of email writing, each of the three chatbots—Meta AI, ChatGPT, and Google Gemini—performed exceptionally well. They were able to generate polite and professional emails for a project extension request, utilizing a template style that allowed for personalization. As a result, all three chatbots earned perfect marks for this task.

  • 4-2. Recipe provision

  • When prompted to provide a recipe for chili, all three chatbots returned accurate and thorough recipes, albeit with some minor variations. A critical distinction emerged in their sourcing practices: both Meta AI and Google Gemini included links to the websites from which the recipes were sourced, enhancing trustworthiness. In contrast, ChatGPT did not provide any sources, which raised concerns regarding recipe authenticity and safety. Therefore, for recipe provision, it is advisable to rely on Meta AI or Gemini due to their credible sourcing.

  • 4-3. News summarization

  • All chatbots were able to quickly generate a bulleted list of the latest news upon request. However, they exhibited inconsistency in sourcing; while ChatGPT and Meta AI provided direct links to the news outlets cited, Gemini mentioned various sites without including links. This makes ChatGPT and Meta AI more reliable options for accurate news summarization owing to their citation practices.

  • 4-4. Math problem-solving

  • For the math problems, each chatbot utilized different methods to tackle the algebra and geometry questions posed. Ultimately, only Meta AI succeeded in delivering a satisfactory final answer for the problems. ChatGPT provided a well-structured solution that was incomplete, whereas Gemini failed to insert numeric values, resulting in a theoretical instead of a definitive answer. Thus, Meta AI stands out as the best option for solving math problems.

  • 4-5. Programming tasks

  • Upon receiving a prompt to create a more complex variant of tic-tac-toe using HTML and JavaScript, both Meta AI and ChatGPT delivered the complete code as requested. However, Google Gemini incorrectly provided CSS in place of HTML, which signifies a lack of understanding in differentiating these two technologies. Therefore, for programming tasks, Meta AI and ChatGPT are preferred due to their accuracy in coding.

  • 4-6. Mock interview performance

  • For the mock interview simulation request, each chatbot effectively simulated a possible interview scenario, providing relevant questions and answers. Although their approaches varied, all three chatbots produced impressive results that could serve as helpful starting points for job seekers preparing for interviews.

  • 4-7. Information sourcing and citation

  • A significant aspect of the chatbots' performance involved their ability to cite sources when providing information. Meta AI and ChatGPT consistently linked to the sources they referenced, reinforcing their reliability. In contrast, while Gemini mentioned sources, it lacked direct links, which raises concerns about the trustworthiness of the information provided. Therefore, for tasks that require sourcing, Meta AI and ChatGPT are preferred due to their adherence to proper citation practices.

5. Conclusion

  • The comparative assessment of Google Gemini, ChatGPT, and Meta AI underlines their individual strengths and areas of superiority across various tasks. Google Gemini is notably strong in coding and creativity, whereas ChatGPT excels in reasoning and clarity, and Meta AI demonstrates robust performance in math and programming tasks. However, limitations were identified — Gemini's citation practices need improvement, and ChatGPT could enhance recipe sourcing reliability. Future prospects in AI chatbots seem promising, focusing on refining features and addressing limitations to improve user experience. To apply these findings practically, users should select based on their specific needs, leveraging unique strengths of each chatbot for tasks ranging from creative writing to solving complex problems, while remaining cautious of the potential inaccuracies in chatbot outputs.

6. Glossary

  • 6-1. Google Gemini [AI Chatbot]

  • Google Gemini is an AI chatbot subscription service that includes the Gemini Ultra 1.0 AI model and 2 terabytes of Google One cloud storage. It excels in coding proficiency and creative tasks.

  • 6-2. OpenAI ChatGPT [AI Chatbot]

  • OpenAI's ChatGPT Plus offers access to GPT-4 and Dall-E 3 models, a unique GPT store for custom versions, and excels in reasoning clarity and ethical decision-making.

  • 6-3. Meta AI [AI Chatbot]

  • Meta AI is noted for its excellent performance in writing emails, providing recipes, solving math problems, and programming tasks, making it the best overall performer in comparative reviews.

  • 6-4. Microsoft Copilot Pro [AI Chatbot Integration]

  • Copilot Pro integrates with Microsoft Office tools and offers similar access to GPT-4 and Dall-E 3 as ChatGPT, though it is not as frequently highlighted in performance comparisons.

7. Source Documents