# Mesa-Extrapolation code ware


The code is organized as follows:

### The main experimental code is located in the experiments directory.
1. evaluate_passkey_retrieval.py is the main code for evaluating the passkey task.
2. evaluate_ppl.py is the main code for the PPL task of evaluating perplexity.
3. evaluate_generation-govreport.py is the main code for evaluating the summary task.
4. evaluate_speed_memory.py is the main code to evaluate inference speed and memory.
5. ablation_passkey_retrieval.py is the main code to conduct ablation experiments about weave PE based methods: including 'weave-v20', 'rerope-ablation', 'leaky-rerope-ablation'.


### The datas directory mainly places the code for generating passkey data and the code for loading data sets.

### conf directory, mainly used to store experimental results.

### The logs directory records the pictures of the paper.

### The methods directory, integrating the Mesa-Extrapolation method and multiple comparison methods.

### The models directory contains the loading code for different models.

### The results directory is mainly used for drawing pictures and presenting experimental results.

### The utils directory contains tools for calling method designs.
the implemented methods including: 'weave-v20', 'rerope', 'leaky-rerope', 'streaming-llm', 'lm-infinite', 'rerope-ablation', 'leaky-rerope-ablation'

### In the test directory,
probe_visualization.py is used to perform probe experiments.

### In the home directory,
1.    re_pred.py is used to perform longbench experiments.
2.    eval.py is used to evaluate experimental results.

### Note that in our code, the 'weave' method represents our Mesa-Extrapolation method.


### We will upload our code ware to GitHub when accepted.

