Quick start

We recommend to use all the commands within a lxplus instance. To beguin, please be in the directory of this repository.

cd AnomalyDetection

From here, we should create a new pipenv environment to install the required packages. This can be easily done via the script we prepared:

source setup.sh

Now, we prepared a module to easily mount the data from the PLT-BRIL detector. To do so, we need to run the following command in a python shell:

from src.data.mounting_tool import MountData
MOUNT_TARGET = "./Files/22"
mounter = MountData(user="YOUR_CMSUSR", password="YOUR_password")
mounter.create_mount(mount_source="brildev1:/brildata/22/", mount_target=MOUNT_TARGET)

This will mount all the files containing fills in 2022 in the directory ./Files/22. And the data will be unmounted when the lxplus instance is closed. Now, the data is ready to be used.

There are two main ways of running the anomaly detection algortihm. One of them is to run a single fill file:

from src.model.searcher import AnomalySearcher
searcher = AnomalySearcher(MOUNT_TARGET)

# Fill to analyze:
FILL_N = 1
result = searcher(FILL_N,
                      return_preprocessed=True,
                      generate_plots=True,
                      save_path = "./Results/22",
                      )

This function will return a dictionary with the results of the analysis. The plots will be saved in the directory ./Results/22. The plots are saved in the following format: FILL_N.png. The preprocessed data will be saved in the following format: ./Results/22/single_fill_reports/FILL_N.pkl.

The other way of running the algorithm is to run it over all the fills in a given directory:

AnomalySearcher().run_scan(
    mount_path=MOUNT_TARGET,
    output_path="./Results/22",
    make_anomalous_plots=True,
    overwrite=False,
    progress_bar=True,
)

Running this function will create a directory ./Results/22 with the following structure:

./Results/22 |
             | -plots
                |  -FILL_0.png
                |  -FILL_1.png
                |  -FILL_2.png
                |   ...
             | -single_fill_reports
             | -report_df.pkl
             |  -logs.json

The report_df.pkl file contains a pandas dataframe with the results of the analysis. The logs.json file contains the logs of the analysis. The plots directory contains the plots of the anomalous fills. The single_fill_reports directory contains the analysis data of the fills, reporting different types of anomalies and their respective timestamps.

Please refer to the examples to see more examples of how to use the code.