Regressive Plasticity Schedule
For a detailed, formal paper, please visit: https://github.com/iamjasonfeng/RPS-Paper I created an LLM post-training method called Regressive Plasticity Schedule (RPS). Preliminary results show that RPS improved Qwen3-8b's ARC-AGI performance and program synthesis reliability. RPS is inspired by neuroscience. As humans, we learn basic skills as kids with high neuro-plasticity. We then learn advanced skills as teens and adults with low neuro-plasticity. RPS trains a model in 2 stages. In stage 1, the model is trained on easy data with high learning rate. In stage 2, the model is trained on hard data with 10% the learning rate of stage 1. RPS is basically a combination of existing ideas: curriculum learning + learning rate decay. Training setup: RPS and EPS Training Setup For these experiments, I used qwen3-8b as the base model and trained it with Alibaba Model Studio managed DPO fine-tuning using LoRA. The goal was to test whether a staged “plasticity schedule” can improve A...