Tech Developments and Innovations in AI and Hardware Demonstrated at COMPUTEX Taipei 2024

1. Summary

• The report, titled 'Tech Developments and Innovations in AI and Hardware Demonstrated at COMPUTEX Taipei 2024,' outlines the key technological advancements presented at the event. Companies like SK hynix, ASUS, Mnemonic Electronic, and GeIL showcased innovations in AI memory solutions, high-capacity SSDs, AI-enabled PC hardware, and advanced cooling systems for memory modules. The report also discusses the rising demand for NVIDIA GPUs in China and Intel's new CPU developments. Furthermore, it highlights the increasing RAM requirements for AI-driven smartphones and the evolving landscape of small phone designs aided by smart assistant capabilities.

2. COMPUTEX Taipei 2024 Highlights

• 2-1. SK hynix's AI Memory Solutions (HBM3E and CXL Memory Modules)

• SK hynix presented its advanced AI memory solutions at COMPUTEX Taipei 2024, reinforcing its status as a leading AI memory provider. The showcase included the HBM3E, which offers data processing speeds of 1.18 TB per second, and the CXL Memory Module-DDR5 (CMM-DDR5) which enhances system bandwidth and processing capacity for AI servers. SK hynix also displayed server DRAM products like DDR5 RDIMM and MCR DIMM, including the tall 128-GB MCR DIMM.

• 2-2. Mnemonic Electronic's High-Capacity SSDs

• Mnemonic Electronic highlighted high-capacity SSDs under the theme "Embracing the Era of High-Capacity SSDs." Notable products included the Mnemonic MS90 8TB SATA SSD, FORESEE ORCA 4836 series enterprise NVMe SSDs, and FORESEE XP2300 PCIe Gen 4 SSDs. These products cater to industrial-grade, automotive-grade, and enterprise-grade storage needs, demonstrating a variety of interfaces such as PCIe M.2, PCIe BGA, SATA M.2, and SATA 2.5-inch.

• 2-3. ASUS's AI PC-Ready Hardware and Prototype Motherboards

• ASUS introduced new AI PC-ready hardware and prototype motherboards, leveraging AI-powered applications and features. The showcased hardware aims to enable users to run AI tools on personal AI PCs, highlighting ASUS's commitment to creating products that support the integration of AI into everyday computing.

• 2-4. GeIL's Memory with Active Dual Fan Cooling System (EVO V series and TUF GAMING ALLIANCE MEMORY)

• GeIL displayed its EVO V series and TUF GAMING ALLIANCE MEMORY featuring an Active Dual Fan Cooling System, which ensures optimal stability and performance. This cooling system enhances heat dissipation, maintaining stability under demanding conditions. Additionally, GeIL’s offerings include RGB SKUs without fans for users seeking aesthetic customization.

• 2-5. Future Memory Technologies (LPDDR6 and CAMM2)

• The evolution of memory technologies like LPDDR6 and CAMM2 was highlighted, focusing on enhanced performance and efficiency. LPDDR6 offers bus speeds of up to 14.4 GT/s, a 50% increase over LPDDR5, and features native ECC support. CAMM2 modules, designed for thin-and-light notebooks, are expected to play a significant role in future desktop and notebook designs, offering higher speeds, larger capacities, and energy efficiency.

3. Technology and Market Trends

• 3-1. Demand for NVIDIA GPUs in China

• NVIDIA's GPUs, particularly the H20 model, have seen massive demand in China. The financial impact of this is substantial, with NVIDIA expecting $12 billion in revenue from its Chinese venture. Over one million H20 GPUs have already been sold, greatly surpassing the 550,000 Ascend 910B chips from Huawei. Despite the Huawei Ascend 910B's higher Total Processing Performance (TPP), the H20 GPU performs better in real-world applications due to its superior memory configuration, including high HBM3 memory bandwidth.

• 3-2. Intel's Arrow Lake CPUs and Xe2 'Battlemage' GPUs

• Intel's upcoming Arrow Lake 'S' Desktop and 'HX' laptop CPUs are being launched without dedicated NPU hardware, limiting NPUs to Arrow Lake-H/U and Lunar Lake chips. AMD currently holds the domain of desktop chips with dedicated NPUs, yet Intel's future refreshes might incorporate NPUs into Arrow Lake-S and Arrow Lake-HX. Moreover, Intel's discrete GPUs, based on Xe2 'Battlemage' architecture, aim to capture a greater market share in gaming. These GPUs, with codename 'Churchill Falls,' are anticipated to offer improved AI and ray tracing performance.

• 3-3. TSMC's FOPLP Technology

• TSMC, since introducing its FOPLP technology in partnership with OSAT providers, aims to offer cost-effective packaging solutions for chips. This endeavor includes transitioning from traditional IC packaging to FOPLP for both AI GPUs and consumer ICs. Companies like AMD have engaged with TSMC to explore these packaging solutions, which are designed to enhance cost efficiency and drive industry interest.

• 3-4. Panmnesia's CXL Solution

• South Korean startup Panmnesia unveiled a low-latency Compute Express Link (CXL) IP to expand GPU memory using PCIe-connected DRAM or SSDs. Known as CXL-Opt, this solution overcomes traditional memory constraints of GPUs, demonstrating significant performance improvements with two-digit nanosecond round-trip latency, a reduction from the older 250 nanoseconds. This advancement points to potential adoption in expanding the memory capacity efficiently, although cost and latency challenges persist.

• 3-5. Opera GX Browser AI Features

• Opera GX has introduced an update to its built-in AI, Aria, through its AI Feature Drops program. This update enhances the browser with image generation capabilities via Google's Imagen2 model, voice output, chat summary options, and source links. These features are aimed at improving the visual and interactive experience of users, facilitating the generation of up to 30 images per day per user based on text prompts.

4. AI-driven Smartphone Requirements

• 4-1. Increasing RAM Requirements due to AI

• AI technology is significantly driving up the RAM requirements in modern smartphones. The integration of AI capabilities into mobile devices demands more memory to facilitate advanced operations. For example, many top-of-the-line Android flagships now come equipped with at least 12GB of RAM to manage next-gen use cases efficiently. This requirement stems from the computationally expensive nature of AI models, which often necessitate instant access to large amounts of fast memory. The Pixel 9 series is expected to debut with 12-16GB RAM to ensure smooth AI functionality.

• 4-2. Significance of Higher RAM Capacities in Future Smartphones

• Higher RAM capacities are becoming increasingly important for supporting AI functionalities in future smartphones. Smartphones with insufficient RAM, such as certain models in the iPhone 15 series with only 6GB, have missed out on first-generation AI features. Implementing AI effectively requires substantial memory, and even heavily compressed models for mobile devices need gigabytes of RAM. As AI features and technologies continue to evolve, having more RAM is crucial for future-proofing devices and ensuring they can handle upcoming AI advancements.

• 4-3. Leading Companies in AI Development for Smartphones

• Major players like Apple, Google, and Samsung are leading the way in AI development for smartphones. These companies are incorporating robust AI features supported by high RAM capacities in their devices. Apple and Google, for instance, are integrating AI into their latest models, ensuring that the necessary hardware is in place for optimal AI performance. While other brands like OnePlus and Xiaomi are also incorporating more RAM into their smartphones, they are still catching up in terms of meaningful AI capabilities, mainly due to the high costs and expertise required for AI software development.

5. The Return of Small Phones

• 5-1. Potential Impact of AI on Small Phone Adoption

• The resurgence of small phones is significantly influenced by advancements in Artificial Intelligence (AI). The presence of highly capable smart assistants, like Google's Gemini and Apple's updated Siri, plays a crucial role. These AI-driven assistants can perform multiple tasks for the user, thus reducing the dependency on larger screens. This shift is anticipated to enable smaller phone designs by offloading various functions to smart assistants, facilitating a user experience that doesn’t solely rely on the display size.

• 5-2. Cloud Offloading Benefits and Smart Assistants

• To further enhance the feasibility of small phones, offloading computational tasks to the cloud is deemed beneficial. By relying on cloud-based processing, phones can operate with smaller batteries and less powerful chipsets, thereby reducing the device's overall size. Although on-device requests are quicker, cloud processing ensures lighter and more efficient phone hardware. This approach is expected to support the advancement of smaller phones while maintaining functionality and performance.

• 5-3. Example of Rabbit R1 Non-Traditional Smartphone Design

• The Rabbit R1, despite being a poorly executed product, exemplifies innovative thinking in small phone design. It features a smaller display and battery, leveraging voice commands instead of traditional on-screen interactions. Although its rotating crown and other features are not practical, the Rabbit R1 points to a future where smartphones can become more compact by minimizing on-device processing and display size. This non-traditional design paradigm hints at potential pathways for the development of truly small, functional phones.

6. Apple’s Integration of AI Features

• 6-1. Apple Intelligence in iOS 18, iPadOS 18, and macOS Sequoia

• Apple introduced Apple Intelligence at WWDC24, an AI that is fully integrated into iOS 18, iPadOS 18, and macOS Sequoia. However, not all devices that can run these operating systems will be able to use Apple Intelligence, as it will only be available on the most powerful ones.

• 6-2. Supported Devices and Initial Availability

• Only specific devices will support Apple Intelligence. The AI is available on iPhone 15 Pro models and iPhone 16 series, including iPhone 15 Pro, iPhone 15 Pro Max, iPhone 16, iPhone 16 Plus, iPhone 16 Pro, and iPhone 16 Pro Max. For Macs, only those with an Apple Silicon chip are compatible, such as MacBook Air M1, M2, and M3, MacBook Pro M1, M2, and M3, iMac M1 and M3, Mac mini M1 and M2, Mac Studio M1 and M2, and Mac Pro M2. As for iPads, compatibility is limited to models with an M-series processor such as iPad Pro M1, M2, and M4, and iPad Air M1 and M2. Initially, Apple Intelligence will only be available in the US.

• 6-3. Exclusion of HomePod and Apple Watch

• Despite the integration of Apple Intelligence into Apple’s operating systems, certain devices will not support the company's AI. Specifically, the HomePod and Apple Watch will not be compatible with Apple Intelligence at launch, although they are compatible with Siri.

7. AI and Computational Challenges

• 7-1. Computational Power for Training GPT-4 Scale Models

• According to the document titled 'AI's $600B Question | Hacker News,' Jensen states that training a 1.8 trillion parameter MoE GPT-4 scale model requires about 8,000 H100 GPUs running for 90 days. Meta reportedly has about 350,000 GPUs, which theoretically enables them to train 50 GPT-4 scale models every 90 days. However, it is noted that only a fraction of these GPUs are linked in a way that allows for large-scale model training, significantly reducing their capacity. The actual number of GPT-4 models Meta can train is closer to six every 90 days. This computational requirement implies a high cost in networking to maintain such a cluster, making scaling up AI model training both challenging and expensive.

• 7-2. Impact on Profit Margins and Commoditization of AI Models

• The document discusses the potential commoditization of core AI models due to their large-scale deployment and free distribution by companies like Microsoft and Meta. This scenario could drive profit margins for AI-centric companies close to zero, making it challenging for investors but beneficial for builders. The proprietary data used for training models is highlighted as a primary factor for output quality, more critical than the model itself, posing both technological and legal challenges as regulatory measures to protect proprietary data are anticipated.

• 7-3. Challenges and Implications of VR Technology Adoption

• The adoption of VR technology faces significant hurdles despite technological advancements. The document notes that people generally prefer real-life experiences over virtual ones, affecting the mainstream acceptance of VR. Factors such as setup complexity, social interaction limitations, and high costs deter widespread use. Although there are advancements in hardware like the Vision Pro and Quest 2 that streamline user experience, VR in its current state remains more niche than mainstream. Rumors suggest upcoming prototypes may bring extraordinary improvements, but until fundamental human preferences are addressed, VR technology is unlikely to gain mass adoption.

8. Conclusion

• The innovations revealed at COMPUTEX Taipei 2024 highlight the significant technological strides made by companies such as SK hynix and ASUS, particularly in the realm of AI and memory solutions. NVIDIA's GPUs continue to dominate the market in China, reflecting a robust industry demand. Intel's developments in CPUs and the ongoing emphasis on increasing RAM capacities for smartphones underscore the pivotal role of AI in shaping future technology. While Apple's new AI features in iOS 18, iPadOS 18, and macOS Sequoia demonstrate a leap forward in user experience, the report also touches on the challenges in scaling AI model training and VR technology adoption. These advancements present both opportunities and hurdles, pointing towards a future where AI will profoundly influence various technology sectors. However, issues like the high cost of GPU clusters for AI training and consumer reluctance towards VR indicate areas needing continued innovation and investment.

9. Glossary

• 9-1. COMPUTEX Taipei 2024 [Event]

• An annual technology exhibition held in Taipei, featuring the latest innovations and products from major companies in the tech industry. Highlights include memory solutions, SSDs, and AI hardware.

• 9-2. SK hynix [Company]

• Known for its high-density memory solutions including HBM3E and CXL memory modules, which contribute significantly to AI and computing advancements.

• 9-3. ASUS [Company]

• Showcased AI-ready PC hardware and prototype motherboards at COMPUTEX Taipei 2024, emphasizing their role in advancing computing technology.

• 9-4. Intel Arrow Lake [Technology]

• A future lineup of CPUs by Intel without dedicated NPU hardware, aimed at improving gaming and computational performance.

• 9-5. Apple Intelligence [Technology]

• A suite of AI-driven features integrated into Apple's iOS 18, iPadOS 18, and macOS Sequoia, enhancing user experience with tools like writing aids and personalized emojis.

• 9-6. NVIDIA GPUs [Product]

• High-performance graphics processing units by NVIDIA, witnessing significant demand in China, indicative of robust market interest.

10. Source Documents

• TechPowerUphttps://www.techpowerup.com/news-tags/CAMM
• TechPowerUphttps://www.techpowerup.com/news-tags/AI
• Why I'm making sure my next smartphone has tons of RAMhttps://www.androidauthority.com/next-phone-more-ram-3456172/
• I thought small phones were dead, but I want AI to bring them backhttps://www.androidauthority.com/small-phones-ai-return-3456725/
• Apple Intelligence: All supported deviceshttps://www.gearrice.com/update/apple-intelligence-all-supported-devices/
• AI's $600B Question | Hacker Newshttps://news.ycombinator.com/item?id=40869461