AI company Anthropic secures Series B funding for developing adaptable, comprehensible, and resilient AI systems.
Anthropic Raises $580 Million for AI Safety and Research
Anthropic, a leading AI safety and research company, has recently secured a Series B round of funding totalling $580 million. The round was led by Sam Bankman-Fried, CEO of FTX, and included participation from notable investors such as Caroline Ellison, Jim McClave, Nishad Singh, Jaan Tallinn, and the Center for Emerging Risk Research (CERR).
With the new funding, Anthropic plans to develop a large-scale experimental infrastructure for AI models. This investment will enable the company to further its progress in understanding the source of pattern-matching behavior in large language models, and mathematically reverse engineering their behavior.
Anthropic has been employing reinforcement learning to improve the properties of large language models, and has developed baseline techniques to make them more "helpful and harmless". The company is also expanding its research into making AI systems more steerable, robust, and interpretable.
To assist other research labs, Anthropic is releasing a new dataset that will help train models that are more aligned with human preferences. The company has already made progress in this area, having published an analysis of sudden changes in performance in large language models and their societal impacts.
The financing will also support Anthropic's efforts to understand the source of pattern-matching behavior in large language models. The company has begun to gain insights into this area, and is continuing to investigate this important issue.
The financier of the Series B round at Anthropic is former Google CEO Eric Schmidt, along with Databricks, Nvidia, Microsoft, Salesforce, and a16z.
Anthropic's work in AI safety and research is crucial in ensuring that the development and deployment of AI systems align with human values and preferences. With the new funding, Anthropic is well-positioned to continue its important work in this field.
Sudden Changes in Performance in Large Language Models
Anthropic has also released an analysis of sudden changes in performance in large language models and their societal impacts. The analysis provides valuable insights into how these changes can impact the behaviour of AI systems, and offers recommendations for how to mitigate their risks.
The analysis highlights the importance of continued research and development in this area, and underscores the need for a collaborative and interdisciplinary approach to AI safety and ethics. Anthropic's work in this area is helping to ensure that the development and deployment of AI systems is safe, beneficial, and aligned with human values.
Overall, Anthropic's research and development efforts are helping to advance the field of AI safety and ethics, and are contributing to a more responsible and ethical approach to the development and deployment of AI technology.
