Bio-Inspired Nanochat
Replacing static transformer weights with fluid, metabolic biological analogs.
A research fork of GPT that implements 'living' weights. It introduces presynaptic fatigue (to prevent repetition), postsynaptic fast-weights (for infinite local context), and structural plasticity (experts that are born and die based on energy usage). Optimized via CMA-ES on a 48-parameter biological search space.