“I would say we’re at the beginning [of interpretability of LLMs] maybe we now like understand 3% of how they work.[…] biology and medicine that’s one of the sets of applications I’m most excited about.[…] There are important risks and there are benefits and that is the technology goes on its exponential the risks become greater and the benefits become greater .[…] Anthropic is very interested in these questions of catastrophic risk. We have this thing called responsible scaling policy and that’s basically about measuring models at each step for catastrophic risk.[…]
What is catastrophic risk?
I would put it in two categories. One is misuse of the models which could include things in the realm of biology or cyber or kind of um you know election operations at scale, things that are really disruptive to society to that misuse would be one bucket and then the other bucket would be autonomous unintended behavior of the model so you know today it might be just you know the model doing something unexpected but increasingly as models act in the world we have to worry about them behaving in ways that you wouldn’t expect.”