In the Technical Tuesday session, Building an AGI Supercomputer, the team from SingularityNET, including prominent figures like Janet Adams, Sergey Shalyapin, Haley Lowy, and Teo Valik, went into the ambitious and innovative efforts to create an Artificial General Intelligence (AGI) supercomputer.
This initiative is poised to revolutionize AI, leveraging the expertise of seasoned professionals and groundbreaking partnerships with companies such as Ecoblox and Fetch.ai.
Initial Hardware and Partnerships
The session kicks off with the announcement of significant milestones: the acquisition of initial hardware and the formation of strategic partnerships. Collaborations with companies like Ecoblox and Fetch.ai underscore the project's momentum.
Learn More: SingularityNET Announces Initial $53M AI Infrastructure Investment to Advance AGI
With Teo Valik, CEO of Ecoblox and the inventor of the GPU, and Sergey Shalyapin, SingularityNET's CTO, at the helm, the design and execution of the world's first AGI supercomputer are well underway.
The Need for New Architectures
The discussion moves to the limitations of current hardware optimized for neural computing. To transcend these limitations, SingularityNET is pioneering neural symbolic programming, a different architecture poised to enhance AI's capabilities. The team's goal is to build the next generation of supercomputing for neural symbolic computing, optimizing large language models (LLMs) with neuro-symbolic methods, and ultimately creating a decentralized singularity for humanity's benefit.
Evolution of AI Research
Sergey Shalyapin discusses the evolution of AI research from optimizing GPU computing workflows for simple tasks to building memory models at large scale that can process sequences of actions and images as sequences. Despite the potential for generalization, current models still operate within the constraints of large datasets and parameter reutilization.
To move beyond these constraints, SingularityNET is exploring progressive paradigms like continual learning and integrating various computing processes, including matrix, vector, and quantum computing. Optimal workflows, workloads, and memory management are emphasized as crucial for developing advanced AI systems.
They note that while these models have the potential for beautiful generalization, they still operate within the same constraint paradigm of training on large datasets and reutilizing parameters.
Scaling AGI Supercomputing
Haley Lowy highlights the excitement within SingularityNET about scaling their AGI supercomputer. With decades of experience in neural networks and neural-symbolic integration, the team has developed innovative products through scalable experiments.
Now, leveraging their expertise in AGI, mathematical optimization, and computing, they are developing the AIISH framework and OpenCog, culminating in a scalable version called Hyperon.
The collaboration with Teo Valik, who shares their technical enthusiasm, has been instrumental in their progress. They have previously worked on NGI and early applied AI for decades, and have been experimenting with small-scale neural networks. The scalability of these experiments led to innovative products that have surpassed expectations.
Teo Valik's Journey and Contributions
Teo Valik recounts his journey from discovering GPUs in 2002 to working with Nvidia and ATI in Silicon Valley, transitioning from gaming applications to developing GPU supercomputers. His career has taken him to Asia, where he contributed to the democratization of computing. Valik's work highlights the importance of flexible hardware and the necessity for engineers to transcend geographic limitations, a testament to the advancements in technology.
He explains how GPUs were 128-bit computers while CPUs were 32-bit, and how he went from using GPUs for gaming to using them for computing and ultimately developing GPU supercomputers. Teo's career took him to developing GPU supercomputers predominantly in Asia, leading to the democratization of computers. He came from Croatia and had to move to Silicon Valley to work on computer chips, but now, with the advancements in technology, leading-edge engineers no longer have to be limited to certain countries.
Mission-Driven Technological Solutions
Teo also discusses SingularityNET's mission to solve problems using technology and eliminate human bias. Examples include automated customs clearance in Dubai and the development of multi-blockchain IDs and universal translators. These solutions require massive data, measured in zettabytes, underscoring the importance of brilliant minds like Janet Adams and Dr. Ben Goertzel in building a better world.
He uses the example of automated customs clearance in Dubai and the need to create a technology stack that eliminates discrimination. SingularityNET is working on solutions such as IDs with multiple blockchains and universal translators.
To commercialize these technologies, massive amounts of data are required, universal in zettabytes. He also reflects on the transformational potential of repurposing technologies and the comparison of Archimedes' invention of the screw and steam power to the current advancements in GPU technology.
Ecoblox and Environmental Concerns
Haley expresses excitement about the innovative work being done in AI and the involvement of brilliant minds at SingularityNET. Haley introduces Ecoblox, a modular data center solution designed to reduce carbon emissions and environmental impact.
These data centers, deployable near Telco towers, solar panels, and underwater internet cables, represent the first-generation ecosystem focusing on CPU and GPU servers for neurosymbolic AI and AGI. Partners like Asus, Gigabyte, AMD, Nvidia, and Tornado are collaborating to create an open standard platform named OpenCog, expected to drive significant innovation and industry focus towards SingularityNET's mission.
Hardware for AGI
The team addresses the critical question of what type of hardware AGI needs. AGI is viewed as a technology stack, a dynamic computing process, and an ecosystem. It requires different types of memory and extensive use of graphs for explicit and implicit representations.
AGI encompasses a wide family of algorithms, functioning as memory arrays, neural networks, and symbolic algorithms, necessitating dynamic retraining cycles. The specific hardware requirements for AGI are detailed further in the session.
Haley discusses the importance of having the right hardware for the race to develop Artificial General Intelligence (AGI). She believes that SingularityNET, a decentralized AI community, now has the necessary hardware to compete and win in this race.
However, the question of what type of hardware AGI needs is not yet fully answered. Sergey explains that AGI can be viewed as a technology stack, a dynamic computing process, and an ecosystem. From a technology perspective, AGI deals with different types of memory and will utilize graphs extensively due to the need for explicit and implicit representations.
AGI is a wide family of algorithms that can operate as memory arrays, neural networks, and symbolic algorithms, requiring dynamic retraining cycles. Sergey aims to further explain the specific hardware requirements for AGI in the following sections.
Complex Technology Stack and Continual Learning
Sergey elaborates on the complex technology stack needed for AGI, focusing on knowledge distillation, pattern matching, and mapping at scale. The importance of continual learning and complex joint learning frameworks is emphasized. A supercomputing framework requires low-latency networks capable of handling various computing paradigms, optimizing all nodes, and calibrating the system for supercomputing workloads.
Sergey discusses the importance of building a powerful and competitive AGI system, including the use of memory models and decentralized AI nodes. They explain that AGI will be a decentralized algorithm and that powerful AI nodes, like Hyperon nodes, will be distributed across the network.
Sergey also mentions that the supercomputing nodes will be part of a wider decentralized network and that different latencies will be taken into account. The goal is to create an elegantly calibrated ecosystem that utilizes different interconnection speeds and efficiently utilized widely distributed systems of computing and memory. He adds that AGI is a non-trivial thing and that building the AGI supercomputer is a huge undertaking, involving the design of high-performance computing clusters and the creation of a decentralized network infrastructure with similar aims.
He also addresses a question about the technical contrasts between building traditional high-performance computing clusters and the decentralized network infrastructure, explaining that SingularityNET is building a hybrid system using heterogeneous hardware to optimize AI workloads.
Decentralized AI Nodes
AGI will be a decentralized algorithm, with powerful AI nodes like Hyperon nodes distributed across the network. This decentralized network will account for different latencies, creating an elegantly calibrated ecosystem. Building the AGI supercomputer involves designing high-performance computing clusters and a decentralized network infrastructure, a monumental task that requires innovation in both hardware and software.
Challenges and Partnerships
Teo discusses the challenges of designing a supercomputer for AGI, emphasizing the importance of flexible hardware. Partnerships with hardware vendors like AMD and Nvidia, along with intelligent networking on SingularityNET's EXA container supercomputers, are crucial.
The goal is to achieve near real-time communication and energy efficiency, delivering compute power where needed, not just in centralized locations.
Teo discusses the challenges of designing a supercomputer for an Artificial General Intelligence (AGI) system and the importance of flexible hardware. He also emphasizes the need for near real-time communication and energy efficiency, and the goal of delivering compute power where it's needed, not just in an ivory tower location. They also touch on the decentralized nature of the compute fabric being built with networks like Nunet and Hypercycle.
Metagraph and Optimized Architectures
The team explores the implications of the metagraph for lower-cost distributed computing, privacy, trust, and open governance. Sergey discusses the importance of improving data quality and hardware for AGI development, working with hardware vendors to create optimized chip architectures. The lengthy process of AGI chip design and development, from ideation to qualification, is highlighted.
Ownership of Infrastructure
Teo emphasizes the importance of owning one's infrastructure in developing decentralized platforms, criticizing reliance on cloud services like Amazon AWS. SingularityNET has partnered with hardware companies to establish its infrastructure, aiming to tackle big tech dominance and win the AGI race.
Sustainable AI and Future Directions
Haley concludes by discussing the shift towards neural-symbolic AI and AGI computing, which reduces resource requirements compared to deep neural network processing. This shift is seen as beneficial for the planet and future generations. Sustainable energy sources and water cooling are part of SingularityNET's strategy. The team is excited about future developments, including the Hyperon project and decentralized hardware stack, and looks forward to continuing these important conversations.
SingularityNET creating an AGI supercomputer is a complex and ambitious effort. It entails innovative technology, calculated alliances, and a dedication to AI's moral and sustainable advancement. The team's goal is to establish a decentralized singularity that serves all of humanity, and their combined experience and inventive energy drive this goal.
About SingularityNET Ambassador Program
SingularityNET Ambassador Program provides an avenue for members to actively participate in the ecosystem's growth and development. Through weekly meetings, guilds, and workgroups, members are kept up to date with the latest developments and initiatives and are given the opportunity to contribute and provide feedback. It is a one-stop shop for everything happening within the SingularityNET ecosystem.