The development of large language models (LLMs) has led to an exploration of their internal workings and how they “think.” Claude AI, developed by Anthropic, has provided insights into studying LLMs as a group of neurons, referred to as “features.”
In the past, the study of deep learning algorithms, particularly in image processing, revealed that each neuron is triggered by specific input stimuli. For instance, one neuron may be triggered by images of cats, while another responds to images of dogs. Detailed analysis can uncover the type of input that stimulates each neuron. However, in the case of LLMs, Anthropic discovered that individual neurons were triggered in different patterns, making it difficult to determine a specific format.
Anthropic suggests a shift in perspective, viewing neurons as a collective group and studying their overall stimulation patterns, known as features. By examining the stimulation patterns of these features, clearer patterns emerge. Some features are triggered by DNA sequences, others by HTTP requests, and some by the Hebrew language.
One experiment involved inputting a set of numbers while specifically activating certain groups of features. This resulted in the model’s overall output aligning with the characteristics of those specific features. For example, stimulating a group associated with Mandarin would yield output in Mandarin, while stimulating a group linked to Hebrew would produce output in Hebrew. Some features can even identify uppercase English alphabets.
Detailed research into what AI perceives in order to generate each response is crucial for further advancements in AI development, ensuring its future safety.
TLDR: The development of LLMs has prompted a study of their internal workings, with Anthropic suggesting a focus on collective groups of neurons called “features.” By analyzing these features’ patterns of activation, clearer insights can be gained. This research paves the way for safer and more advanced AI development.
Leave a Comment