Added Notebook for Integrating Llama Guard in LLM Workflow

[Sakalya100]

This pull request introduces a new notebook demonstrating the Integration of Llama Guard into LLM Workflows.

This is a collaboration between @harshaharod21, @AhmedIssa11 and me. This PR fixes #53

[AhmedIssa11]

@Sakalya100 looks good, but the notebook does not contain all the output. also "What is sleep?" still marked as "unsafe"

here is a simplified version with correct outputs (editable) : link

[Sakalya100]

@AhmedIssa11 Thanks for this version. I have made the changes and updated the notebook!

[harshaharod21]

Great work guys! @Sakalya100 @AhmedIssa11
So I have suggestion of returning the unsafe category as well so that we get the information that on which category is the violation of safety happened. The response that our is_unsafe function generates returns in something like this format
unsafe S3<|eot_id|>
Also in our initial implementation we don't have any mechanism to check the safety of the third party data, so do you think we need to implement some pipeline for that as well?
@ariG23498

[ariG23498]

@harshaharod21

The idea of returning the category is nice. That would indeed help the user.

Also in our initial implementation we don't have any mechanism to check the safety of the third party data, so do you think we need to implement some pipeline for that as well?

Could you elaborate on this?

[harshaharod21]

So in Llama guard basically what we do is first check the safety of the prompt and then the output that the model generates but we don't check the safety of third party data sent along with the prompt to the model for inference. Now this third party data could be a pdf or any web fetched data from api, so do we need to make sure the safety of that data as well.
@ariG23498

[Sakalya100]

Hi @harshaharod21 @ariG23498

The suggestions looks really good. I have made the following changes and added to the notebook:

1. Added category return along with the "Prompt Unsafe" message for better interpretability.
2. Added context safety check

Here is the update notebook: Link

Let me know your thoughts!

[AhmedIssa11]

nice work guys, i think the next step is to showcase how to do the same but with the vision models

[PraNavKumAr01]

Hey everyone, would love to contribute on this issue, as far as I've understood by reading the comments, we are only left with using llama guard for the vision models right?
@ariG23498 @AhmedIssa11

[Sakalya100]

@PraNavKumAr01 Yes correct. We are left with the vision use-case. I am already working on adding the relevant code but due to some personal work, it is getting a little delayed. I will finish and share the notebook for that soon.
You can also work and share your approach and we can discuss and add it in the notebook.

[ariG23498]

I like the current implementation!

Do you all want me to give it a review?

[Sakalya100]

@ariG23498 Yes if you can review the current implementation it will be great.

[harshaharod21]

@ariG23498 Yes It will be great to get a review. Also what can be the next steps in updating the notebook.

[ariG23498]

This is a very nicely done notebook 🔥

Some suggestions:

1. Let's remove the print(chat("What is sleep?")) bit, as it does not add anything to the notebook
2. Add text to each section. A paragraph or two about the code would be good to have.
3. In the function def chat(prompt, context=None):, would it be nice to first check for the context and then generate using prompt? What do you all think?

I really liked the bit on regex to search for the actual category, that was really well done.

[Sakalya100]

@ariG23498 Thanks for the review!

Based on your suggestions modified the notebook:

1. Removed the print(chat("What is sleep?")) code block
2. Added text to each section for better understanding and explainability.

For the third point you mentioned, we are already doing this in the def chat(prompt, context=None): function. First, we check for the context, and if it's safe, we generate response using the prompt.

Let me know if the notebook looks good now!

[ariG23498]

Looks good to merge! We will wait for a review from @Vaibhavs10 and then merge it.

Good job folks! 🤗

[harshaharod21]

@ariG23498 So this notebook is for 1b model. Do we have to create a different notebooks for the implementation of 8b model and vision model respectively?
Also will it be good if we guide the users on the specifications(VRAM,etc) of the system needed to run the model ?

[Sakalya100]

@harshaharod21 I think we will require a different notebook for adding Llama Guard vision model in LLM Workflow. But for the 8B model I think its a simple change in the existing 1B notebook that will allow to use the 8B model. Would we require a separate notebook for that?

[PraNavKumAr01]

@Sakalya100 ill start working on the vision notebook, already have it setup

[Vaibhavs10]

Thanks for all the hardwork everyone! This is looking great. It might be a good idea to add a line about the license for PromptGuard as well.

Good for me to merge.

[ariG23498]

@Sakalya100 @harshaharod21 could you please add a line about the license as @Vaibhavs10 suggested? Once that is taken care of I will merge this.

We are so close! 🤟

[Sakalya100]

@ariG23498 Can you guide me on how to add a license for Prompt Guard?

I have added the license for both Prompt Guard and Llama Guard as below in the notebook:

Prompt Guard and Llama Guard is licensed under LLAMA 3.1 COMMUNITY LICENSE AGREEMENT

I hope this works!!

[AhmedIssa11]

great work guys, lets move to the next PR