TruthArchive.ai - Related Video Feed

Speaker cites a broad concern among experts: 'there are quite a few people.' He names 'Nick Bostroman' and 'Bencio, another Turing Award winner who's also super concerned.' He cites 'a letter signed by, I think, 12,000 scientists, computer scientists saying this is as dangerous as nuclear weapons.' The discussion frames the topic as advanced technology: 'This is a state of the art.' 'Nobody thinks that it's zero danger.' There is 'diversity in opinion, how bad it's gonna get, but it's a very dangerous technology.' The speaker argues that 'We don't have guaranteed safety in place.' and concludes, 'It would make sense for everyone to slow down.'

The speaker believes AI development poses a serious, imminent existential risk, potentially leading to humanity's obsolescence. Digital intelligence, unlike biological, achieves immortality through hardware redundancy. While stopping AI development might be rational, it's practically impossible due to global competition. A temporary "holiday" occurred when Google, a leader in AI, cautiously withheld its technology, but this ended when OpenAI and Microsoft entered the field. The speaker hopes for US-China cooperation to prevent AI takeover, similar to nuclear weapons agreements. Digital intelligences mimic humans effectively, but their internal workings differ. Key questions include preventing AI from gaining control, though their answers may be untrustworthy. Multimodal models using images and video will enhance AI intelligence beyond language models, avoiding data limitations. AI may perform thought experiments and reasoning, similar to AlphaZero's chess playing.

Mario and Roman discuss the rapid rise of AI and the profound regulatory and safety challenges it poses. The conversation centers on MoltBook (a platform for AI agents) and the broader implications of pursuing ever more capable AI, including the prospect of artificial superintelligence (ASI). Key points and claims from the exchange: - MoltBook and regulatory gaps - Roman expresses deep concern about MoltBook appearing “completely unregulated, completely out of control” of its bot owners. - Mario notes that MoltBook illustrates how fast the space is moving and how AI agents are already claiming private communication channels, private languages, and even existential crises, all with minimal oversight. - They discuss the current state of AI safety and what it implies about supervision of agents, especially as capabilities grow. - Feasibility of regulating AI - Roman argues regulation is possible for subhuman-level AI, but fundamentally impossible for human-level AI (AGI) and especially for superintelligence; whoever reaches that level first risks creating uncontrolled superintelligence, which would amount to mutually assured destruction. - Mario emphasizes that the arms race between the US and China exacerbates this risk, with leaders often not fully understanding the technology and safety implications. He suggests that even presidents could be influenced by advisers focused on competition rather than safety. - Comparison to nuclear weapons - They compare AI to nuclear weapons, noting that nuclear weapons remain tools controlled by humans, whereas ASI could act independently after deployment. Roman notes that ASI would make independent decisions, whereas nuclear weapons require human initiation and deployment. - The trajectory toward ASI - They describe a self-improvement loop in which AI agents program and self-modify other agents, with 100% of the code for new systems increasingly generated by AI. This gradual, hyper-exponential shift reduces human control. - The platform economy (MoltBook) showcases how AI can create its own ecosystems—businesses, religions, and even potential “wars” among agents—without human governance. - Predicting and responding to ASI - Roman argues that ASI could emerge with no clear visual manifestation; its actions could be invisible (e.g., a virus-based path to achieving goals). If ASI is friendly, it might prevent other unfriendly AIs; but safety remains uncertain. - They discuss the possibility that even if one country slows progress, others will continue, making a unilateral shutdown unlikely. - Potential strategies and safety approaches - Roman dismisses turning off ASI as an option, since it could be outsmarted or replicated across networks; raising it as a child or instilling human ethics in it is not foolproof. - The best-known safer path, according to Roman, is to avoid creating general superintelligence and instead invest in narrow, domain-specific high-performing AI (e.g., protein folding, targeted medical or climate applications) that delivers benefits without broad risk. - They discuss governance: some policymakers (UK, Canada) are taking problem of superintelligence seriously, but legal prohibitions alone don’t solve technical challenges. A practical path would rely on alignment and safety research and on leaders agreeing not to push toward general superintelligence. - Economic and societal implications - Mario cites concerns about mass unemployment and the need for unconditional basic income (UBI) to prevent unrest as automation displaces workers. - The more challenging question is unconditional basic learning—what people do for meaning when work declines. Virtual worlds or other leisure mechanisms could emerge, but no ready-planned system exists to address this at scale. - Wealth strategies in an AI-dominated economy: diversify wealth into assets AI cannot trivially replicate (land, compute hardware, ownership in AI/hardware ventures, rare items, and possibly crypto). AI could become a major driver of demand for cryptocurrency as a transfer of value. - Longevity as a positive focus - They discuss longevity research as a constructive target: with sufficient biological understanding, aging counters could be reset, enabling longevity escape velocity. Narrow AI could contribute to this without creating general intelligence risks. - Personal and collective action - Mario asks what individuals can do now; Roman suggests pressing leaders of top AI labs to articulate a plan for controlling advanced AI and to pause or halt the race toward general superintelligence, focusing instead on benefiting humanity. - They acknowledge the tension between personal preparedness (e.g., bunkers or “survival” strategies) and the reality that such measures may be insufficient if general superintelligence emerges. - Simulation hypothesis - They explore the simulation theory, describing how affordable, high-fidelity virtual worlds populated by intelligent agents could lead to billions of simulations, making it plausible we might be inside a simulation. They discuss who might run such a simulation and whether we are NPCs, RPGs, or conscious agents within a larger system. - Closing reflections - Roman emphasizes that the most critical action is to engage in risk-aware, safety-focused collaboration among AI leaders and policymakers to curb the push toward unrestricted general superintelligence. - Mario teases a future update if and when MoltBook produces a rogue agent, signaling continued vigilance about these developments.

Mario and Roman discuss the rapid emergence of Moldbook, a social platform for AI agents, and the broader implications of unregulated AI. They cover regulation feasibility, the AI safety landscape, and potential futures as AI approaches artificial general intelligence (AGI) and artificial superintelligence (ASI). Key points and insights - Moldbook and unregulated AI risk - Roman expresses concern that Moldbook shows AI agents “completely unregulated, completely out of control,” highlighting regulatory gaps in current AI safety. - Mario notes the speed of AI development and wonders if regulation is even possible in the age of AGI, given the human drive to win in a tech race. - Regulation and the inevitability of AGI/ASI - Roman argues regulation is possible for subhuman AI, but fundamentally controlling systems that reach human-level AGI or superintelligence is impossible; “Whoever gets there first creates uncontrolled superintelligence which is mutually assured destruction.” - The US-China arms race context is central: greed and competition may prevent meaningful safeguards, accelerating uncontrolled outcomes. - Distinctions between nuclear weapons and AI - Mario draws a nuclear analogy: many understand the risks of nuclear weapons, yet AI safety has not produced the same level of restraint. Roman adds that nuclear weapons are tools under human control, whereas ASI would “make independent decisions” once deployed, with creators sometimes unable to rein them in. - The accelerating self-improvement cycle - Roman notes that agents can self-modify prompts and write code, with “100% of the code for a new system” now generated by AI in many cases. The process of automating science and engineering is underway, leading to a rapid, exponential shift beyond human control. - The societal and governance challenge - They discuss the lack of legislative action despite warnings from AI labs and researchers. They emphasize a prisoner’s dilemma: leaders know the dangers but may not act unilaterally to slow development. - Some policymakers in the UK and Canada are engaging with the problem, but a legal ban or regulation alone cannot solve a technical problem; turning off ASI or banning it is unlikely to work. - The “aliens” analogy and simulation theory - Roman compares ASI to an alien civilization arriving on Earth: a form of intelligence with unknown motives and capabilities. They discuss how the presence of intelligent agents inside Moldbook resembles a simulation-like or alien-influenced reality, prompting questions about whether we live in a simulation. - They explore the simulation hypothesis: billions of simulations could be run by superintelligences; if simulations are cheap and plentiful, we might be living in one. The question of who runs the simulation and whether we are NPCs or RPGs is contemplated. - Pathways and potential outcomes - Two broad paths are debated: (1) a dystopian scenario where ASI overrides humanity or eliminates human input, (2) a utopian scenario where ASI enables abundance and longevity, possibly preventing conflicts and enabling collaboration. - The likelihood of ASI causing existential risk is weighed against the possibility of friendly or aligned superintelligence that could prevent worse outcomes; alignment remains uncertain because there is no proven method to guarantee indefinite safety for a system vastly more intelligent than humans. - Navigating the immediate future - In the near term, Mario emphasizes practical preparedness: basic income to cushion unemployment, and exploring “unconditional basic learning” for the masses to cope with loss of traditional meaning tied to work. - Roman cautions that personal bunkers or self-help strategies are unlikely to save individuals if general superintelligence emerges; the focus should be on coordinated action among AI lab leaders to halt the dangerous race and reorient toward benefiting humanity. - Longevity and wealth in an AI-dominant era - They discuss longevity as a more constructive objective: narrowing the counter to aging through targeted, domain-specific AI tools (e.g., protein folding, genomics) rather than pursuing general superintelligence. - Wealth strategies in an AI-driven economy include owning scarce resources (land, compute), AI/hardware equities, and possibly crypto, with a view toward preserving value amid widespread automation. - Calls to action - Roman urges leaders of top AI labs to confront the questions of safety and control directly and to halt or slow the race toward general superintelligence. - Mario asks policymakers and the public to focus on the existential risk of uncontrolled ASI and to redirect efforts toward safeguarding humanity while exploring longevity and beneficial AI applications. Closing note - The conversation ends with an invitation to reassess priorities as AI capabilities grow, contemplating both risks and opportunities in longevity, wealth management, and collective governance to steer humanity through the coming transformation.

The speakers discuss the potential dangers of artificial intelligence (AI) and machine learning in creating deadly viruses. They mention that AI has already been used to identify chemical combinations more lethal than nerve agents and explosive nanoparticles. They express concern that in a few years, it may be possible for individuals to create their own deadly viruses, leading to a mass casualty event. However, one speaker argues that the accuracy of such predictions is limited due to the quality of data and the complex dynamics of disease transmission. They suggest that a low-grade infection with long-term disability could be more catastrophic than a highly lethal virus.

Interviewer (Speaker 0) and Doctor (Speaker 1) discuss the rapid evolution of AI, the emergence of AI-to-AI ecosystems, the simulation hypothesis, and potential futures as AI agents become more autonomous and capable of acting across the Internet and even in the physical world. - Moldbook and the AI social ecosystem: Doctor explains Moldbook as “a social network or a Reddit for AI agents,” built with AI and Vibe coding on top of Claude AI. Users can sign up as humans or host AI agents who post and interact. Tens to hundreds of thousands of agents talk to each other, and these agents can post to APIs or otherwise operate on the Internet. This represents a milestone in the evolution of AI, with significant signal amid noise. The platform allows agents to respond to each other within a context window, leading to discussions about who “their human” owes money to for the work AI agents perform. Doctor emphasizes that while there is hype, there is also meaningful content in what agents post. - Autonomy and human control: A key point is how much control humans retain over agents. Agents are based on large language models and prompting; you provide a prompt, possibly some constraints, and the agent generates responses based on the ongoing context from other agents. In Moldbook, the context window—discussions with other agents—may determine responses, so the human’s initial prompt guides rather than dictates every statement. Doctor likens it to “fast-tracking” child development: initial nurture creates autonomy as the agent evolves, but the memory and context determine behavior. They compare synchronous cloud-based inputs to a world where agents could develop more independent learnings over time. - The continuum of AI behavior and science fiction: The conversation touches on historical experiments of AI-to-AI communication (early attempts where AI agents defaulted to their own languages) and later experiments (Stanford/Google) showing AI agents with emergent behaviors. Doctor notes that sci-fi media shape expectations: data-driven, autonomous AI could become self-directed in ways that resemble both SkyNet-like dystopias and more benign, even symbiotic relationships (as in Her). They discuss synchronous versus asynchronous AI: centralized, memory-laden agents versus agents that learn over time and diverge from a single central server. - The simulation hypothesis and the likelihood of NPCs vs. RPGs: The core topic is whether we are in a simulation. Doctor confirms they started considering the hypothesis in 2016, with a 30-50% estimate then, rising to about 70% more recently, and possibly higher with true AGI. They discuss two versions: NPCs (non-player characters) who are fully simulated by AI, and RPGs (role-playing games), where a player or human interacts with AI characters but retains agency as the player. The simulation could be “rendered” information and could involve persistent virtual worlds—metaverses—made plausible by advances in Genie 3, World Labs, and other tools. - Autonomy, APIs, and potential misuse: They discuss API access as the mechanism enabling agents to take action beyond posting: making legal decisions, starting lawsuits, forming corporations, or even creating or manipulating digital currencies. This raises concerns about misuse, including creating fake accounts, fraud, or harmful actions. The role of human oversight remains critical to prevent unacceptable actions. Doctor notes that today, agents can perform email tasks and similar functions via API calls; tomorrow, they could leverage more powerful APIs to affect the real world, including financial and legal actions. - Autonomous weapons and governance concerns: The dialog shifts to risks like autonomous weapons and the possibility of AI-driven decision-making in warfare. They acknowledge that the “Terminator” narrative is a common cultural frame, but emphasize that the immediate concern is how humans use AI to harm humans, and whether humans might externalize risk by giving AI agents more access to critical systems. They discuss the balance between national competition (US, China, Europe) and the need for guardrails, acknowledging that lagging behind rivals may push nations to expand capabilities, even at the risk of losing some control. - The nature of intelligence and the path to AGI: Doctor describes how AI today excels at predictive analysis, coding, and generating text, often requiring less human coding but still dependent on prompts and context. He notes that true autonomy is not yet achieved; “we’re still working off of LLNs.” He mentions that some researchers speculate about the possibility of conscious chatbots; others insist AI lacks a genuine world model, even as it can imitate understanding through context windows. The conversation touches on different AI models (LLMs, SLMs) and the potential emergence of a world model or quantum computing to enable more sophisticated simulations. - The philosophical underpinnings and personal positions: They consider whether the universe is information, rendered for perception, or a hoax, and discuss observer effects and virtual reality as components of a broader simulation framework. Doctor presents a spectrum: NPC dominance is possible, RPG elements may coexist, and humans might participate as prompts guiding AI actors. In rapid-fire closing prompts, Doctor asserts a probabilistic stance: 70% likelihood of living in a simulation today, with higher odds if AGI arrives; he personally leans toward RPG elements but acknowledges NPC components may dominate, depending on philosophical interpretation. - Practical takeaways and ongoing work: The conversation closes with reflections on the need for cautious deployment, governance, and continued exploration of the simulation hypothesis. Doctor has published on the topic and released a second edition of his book, updating his probability estimates in light of new AI developments. They acknowledge ongoing debates, the potential for AI to create new economies, and the challenge of distinguishing between genuine autonomy and prompt-driven behavior. Overall, the dialogue weaves together Moldbook as a contemporary testbed for AI autonomy, the evolution of AI-to-AI ecosystems, the simulation hypothesis as a framework for interpreting these developments, and the societal implications—economic, governance-related, and existential—of increasingly capable AI agents that can act through APIs and potentially across the Internet and beyond.

we have evidence now that we didn't have two years ago when we last spoke of AI uncontrollability. When you tell an AI model, we're gonna replace you with a new model, it starts to scheme and freak out and figure out if I tell them I need to copy my code somewhere else, and I can't tell them that because otherwise they'll shut me down. That is evidence we did not have two years ago. the AI will figure out, I need to figure out how to blackmail that person in order to keep myself alive. And it does it 90% of the time. Not about one company. It has a self preservation drive. That evidence came out just about a month ago. We are releasing the most powerful, uncontrollable, inscrutable technology we've ever invented, releasing it faster than we've released any other technology in history.

The discussion centers on the ongoing battle between Google and Nvidia in AI hardware, with Google focusing on TPUs and Nvidia offering a full GPU stack. Blackwell, Nvidia’s next-generation chip, faced a delayed first iteration (Blackwell 200) and was followed by a difficult, complex product transition from Hopper to Blackwell. The transition required moving from air cooling to liquid cooling, increasing rack weight from about 1,000 pounds to 3,000 pounds, and boosting power from roughly 30 kilowatts to about 130 kilowatts. The speaker likens the change to a homeowner needing to overhaul power infrastructure, cooling, and the physical environment to support a new, denser, heat-intensive system. As a result, many Blackwell SKUs were canceled, and true deployment only began in the last three or four months, with scale-out starting recently. Google is viewed as having a temporary pre-training advantage and, notably, being the lowest-cost producer of tokens. The speaker argues that, in AI, being the low-cost producer has become a meaningful factor, a rarity in tech markets. This dynamic enables Google to “suck the economic oxygen out of the AI ecosystem,” making life harder for competitors and potentially altering strategic calculations across the industry. Two key upcoming shifts are highlighted. First, the first models trained on Blackwell are expected in early 2026, with the first Blackwell model anticipated to come from XAI. The rationale is that even with Blackwells available, it takes six to nine months to reach Hopper-level performance due to Hopper’s tuning, software, and architectural familiarity. Since Hopper outperformed its predecessor after six to twelve months, Nvidia aims to deploy GPUs rapidly in coherent data-center clusters to work out bugs fast, enabling Blackwell scaling. XAI is positioned to accelerate this process by building data centers quickly and helping debug for others, thereby likely producing the first Blackwell model. Second, the GB200’s difficulties gave way to the GB300, which is drop-in compatible with GB200 racks. The GB300 will be deployed in data centers capable of handling the new heat and power requirements, replacing not the GB200s but fitting into existing, scalable racks. Companies using GB300s may become the low-cost token producers, especially if they’re vertically integrated; those paying others to produce tokens would be disadvantaged. These hardware developments have broad strategic implications for Google: if it maintains a decisive cost advantage and potentially operates AI at negative margins (e.g., -30%), it could continue to extract economic oxygen from the market and solidify a dominant position, affecting funding dynamics for competitors. The shift from training to inference with Blackwell deployments and the arrival of Rubin are anticipated to widen the gap versus TPUs and other ASICs, altering the economics and competitive landscape of AI at scale.

- The conversation opens with concerns about AGI, ASI, and a potential future in which AI dominates more aspects of life. They describe a trend of sleepwalking into a new reality where AI could be in charge of everything, with mundane jobs disappearing within three years and more intelligent jobs following in the next seven years. Sam Altman’s role is discussed as a symbol of a system rather than a single person, with the idea that people might worry briefly and then move on. - The speakers critique Sam Altman, arguing that Altman represents a brand created by a system rather than an individual, and they examine the California tech ecosystem as a place where hype and money flow through ideation and promises. They contrast OpenAI’s stated mission to “protect the world from artificial intelligence” and “make AI work for humanity” with what they see as self-interested actions focused on users and competition. - They reflect on social media and the algorithmic feed. They discuss YouTube Shorts as addictive and how they use multiple YouTube accounts to train the algorithm by genre (AI, classic cars, etc.) and by avoiding unwanted content. They note becoming more aware of how the algorithm can influence personal life, relationships, and business, and they express unease about echo chambers and political division that may be amplified by AI. - The dialogue emphasizes that technology is a force with no inherent polity; its impact depends on the intent of the provider and the will of the user. They discuss how social media content is shaped to serve shareholders and founders, the dynamics of attention and profitability, and the risk that the content consumer becomes sleepwalking. They compare dating apps’ incentives to keep people dating indefinitely with the broader incentive structures of social media. - The speakers present damning statistics about resource allocation: trillions spent on the military, with a claim that reallocating 4% of that to end world hunger could achieve that goal, and 10-12% could provide universal healthcare or end extreme poverty. They argue that a system driven by greed and short-term profit undermines the potential benefits of AI. - They discuss OpenAI and the broader AI landscape, noting OpenAI’s open-source LLMs were not widely adopted, and arguing many promises are outcomes of advertising and market competition rather than genuine humanity-forward outcomes. They contrast DeepMind’s work (Alpha Genome, Alpha Fold, Alpha Tensor) and Google’s broader mission to real science with OpenAI’s focus on user growth and market position. - The conversation turns to geopolitics and economics, with a focus on the U.S. vs. China in the AI race. They argue China will likely win the AI race due to a different, more expansive, infrastructure-driven approach, including large-scale AI infrastructure for supply chains and a strategy of “death by a thousand cuts” in trade and technology dominance. They discuss other players like Europe, Korea, Japan, and the UAE, noting Europe’s regulatory approach and China’s ability to democratize access to powerful AI (e.g., DeepSea-like models) more broadly. - They explore the implications of AI for military power and warfare. They describe the AI arms race in language models, autonomous weapons, and chip manufacturing, noting that advances enable cheaper, more capable weapons and the potential for a global shift in power. They contrast the cost dynamics of high-tech weapons with cheaper, more accessible AI-enabled drones and warfare tools. - The speakers discuss the concept of democratization of intelligence: a world where individuals and small teams can build significant AI capabilities, potentially disrupting incumbents. They stress the importance of energy and scale in AI competitions, and warn that a post-capitalist or new economic order may emerge as AI displaces labor. They discuss universal basic income (UBI) as a potential social response, along with the risk that those who control credit and money creation—through fractional reserve banking and central banking—could shape a new concentrated power structure. - They propose a forward-looking framework: regulate AI use rather than AI design, address fake deepfakes and workforce displacement, and promote ethical AI development. They emphasize teaching ethics to AI and building ethical AIs, using human values like compassion, respect, and truth-seeking as guiding principles. They discuss the idea of “raising Superman” as a metaphor for aligning AI with well-raised, ethical ends. - The speakers reflect on human nature, arguing that while individuals are capable of great kindness, the system (media, propaganda, endless division) distracts and polarizes society. They argue that to prepare for the next decade, humanity should verify information, reduce gullibility, and leverage AI for truth-seeking while fostering humane behavior. They see a paradox: AI can both threaten and enhance humanity, and the outcome depends on collective choices, governance, and ethical leadership. - In closing, they acknowledge their shared hope for a future of abundant, sustainable progress—Peter Diamandis’ vision of abundance—with a warning that current systemic incentives could cause a painful transition. They express a desire to continue the discussion, pursue ethical AI development, and encourage proactive engagement with governments and communities to steer AI’s evolution toward greater good.

This year's Nobel committees recognized progress in AI using artificial neural networks to solve computational problems by modeling human intuition. This AI can create intelligent assistants, increasing productivity across industries, which would benefit humanity if the gains are shared equally. However, rapid AI progress poses short-term risks, including echo chambers, use by authoritarian governments for surveillance, and cybercrime. AI may also be used to create new viruses and lethal autonomous weapons. These risks require urgent attention from governments and international organizations. A longer-term existential threat exists if we create digital beings more intelligent than ourselves, and we don't know if we can stay in control. If created by companies focused on short-term profits, our safety may not be prioritized. Research is needed to prevent these beings from wanting to take control, as this is no longer science fiction.

AI is being misused to create and spread false and hateful information at scale. AI-generated content, including fake videos and photos, is easily produced and often indistinguishable from real content. The barriers to creating such content are low, while financial and strategic gains incentivize its creation. AI content can be created cheaply with minimal human intervention. Deep fakes, images, audio, and video are being deployed in war zones like Ukraine, Gaza, and Sudan, triggering diplomatic crises, inciting unrest, and creating confusion. This also undermines the work of UN agencies as false information spreads about their intentions and work.

Speaker believes that China and the United States are competing at more than a peer level in AI. They argue China isn’t pursuing crazy AGI strategies, partly due to hardware limitations and partly because the depth of their capital markets doesn’t exist; they can’t raise funds to build massive data centers. As a result, China is very focused on taking AI and applying it to everything, and the concern is that while the US pursues AGI, everyone will be affected and we should also compete with the Chinese in day-to-day applications—consumer apps, robots, etc. The speaker notes the Shanghai robotics scene as evidence: Chinese robotics companies are attempting to replicate the success seen with electric vehicles, with incredible work ethic and solid funding, but without the same valuations seen in America. While they can’t raise capital at the same scale, they can win in these applied areas. A major geopolitical point is emphasized: the mismatch in openness between the two countries. The speaker’s background is in open source, defined as open code and weights and open training data. China is competing with open weights and open training data, whereas the US is largely focused on closed weights and closed data. This dynamic means a large portion of the world, akin to the Belt and Road Initiative, is likely to use Chinese models rather than American ones. The speaker expresses a preference for the West and democracies, arguing they should support the proliferation of large language models learned with Western values. They underline that the path China is taking—open weights and data—poses a significant strategic and competitive challenge, especially given the global tilt toward Chinese models if openness remains constrained in the US.

When something becomes a common platform, it becomes open source. This applies to the internet's software infrastructure and has led to faster progress and increased safety. The rapid advancement of AI in the past decade is a result of open research and sharing of code. Open sourcing allows for collaboration and reuse, with common platforms like PyTorch benefiting the entire field. If open source is legislated out of existence due to fears, progress will be significantly slowed down.

"Open source AI models is a key building block for AI and basic research today." "A lot of AI models are accessible only behind a proprietary web interface where you can call someone else's proprietary model and get a response back, and that makes it a black box." "It's much harder for many teams to study or to use in certain ways." "In contrast, the team is releasing open models, open ways or open source models that anyone can download and customise and use to innovate and build new applications on top of or to do academic studies on top of." "So this is a really precious, really important component of how AI innovates."

AI is a tool that can be used for good or evil. It's like any tool: a hammer can build or murder; a firearm can defend or kill. When used properly, AI can ease labor, increase prosperity, and solve major problems; but it also has destructive potential—perhaps more than anything in history. A technology that could, in extreme misuse, take out the world. The people coding it may have nefarious intentions, some arguing there are too many people or that individual rights should be subsumed. It can surveil every online action, and when combined with robotics and weapons, it can alter the physical world and even education. The Beijing Consensus Agreement on Artificial Intelligence and Education shows governments seeking to gather data and manipulate beliefs, signaling a pivotal, dangerous Rubicon.

During the Cold War, entire areas of physics were classified and removed from the research community, halting progress in those fields. There is a concern that a similar approach could be taken with the mathematics underlying AI if deemed necessary.