According to a Pinwan report on September 5, Nous Research teamed up with researchers from EleutherAI Laboratory and the University of Geneva in Switzerland to publish a paper and announced a method called YaRN that can expand the context window of large models. The paper shows that YaRN requires fewer labeling and training steps than traditional methods. Utilizing this approach, LLaMA models can efficiently leverage and extrapolate to longer contexts than their original pre-training allowed, while exceeding the state-of-the-art of past window extensions. According to the GitHub page, LLaMA 2 7B/13B context windows fine-tuned with YaRN can reach 64k and 128k respectively.