'Let OddRam Protect!' OddRam Logo

OddRam

[OddRam is a *space* created for collecting OpenDawg's research on Guardrails, Safety, Ethics & Morals for LLMs.]

Although LLMs were initially issued without particular attention to safety, the release of widely used apps has exposed this newly perceived flaw.

Even notes around fairly recent releases of LLama models suggest it is up to the LLM app developers to add guardrails, such as LLamaGuard[1], rather than the core model trainer.

Although "Instruct" model variations generally have had some additional training and tuning to encourage it to become a 'helpful assistant', this is often not sufficient to shut down queries about unsafe, unsavoury or outright illegal subjects.

Whilst researching; we have identified some of the possible approaches currently available to the model releasing/using community:

  1. Fix The Training data.
    New models may have extensive "cleaning" of training data sets to remove unwanted inputs. This may be done manually or synthetically, probably mostly the latter. Essentially trying to mitigate the 'garbage in garbage out' syndrome.
  2. Guards.
    The LLamaGuard[1] paper makes reference to Anthropic's safety training research[2], and by using Anthropic's 'Red book'[3]* & other data the researchers fine-tune trained their own LLM to help warn users (and App developers) of 'unsafe' conversations. This approach does, however, have the issue of requiring multiple LLMs to be running. As newer models are developed they could add specific Guard layers in a similar way to how they add the image or audio functionality layers. Initially, some issues were not addressed around multi-language safety as early data sets were predominantly in English. Nemotron's CultureGuard[4] is an interesting attempt to address this.
  3. Programmatic Solutions.
    There are more traditional programmatic methods of cleaning language and 'hot' words, but not using AI. e.g. using 'clean lists' of approved words or checking 'black lists' of bad language to be scrubbed.
  4. Context injections.
    As maximum context window sizes expand and the major AI players start to monetize via 'token burn', injecting 'few shot' style training within the context window can both rapidly direct conversations away from hot topics and also reduce unwanted output. Although token costs may increase, it is certainly easier than running fine-tuning training repeatedly each time a model gets updated. This 'context injection' can be seen as a 'Few Shot' approach, but that term could also apply to a very limited LoRA training session.
  5. Zero shot.
    Your user query could include specific instructions and examples to guide the conversation response. Although this user 'self-safety' approach may be helpful to get answers with specific morals & ethics considered, it will likely not protect unwary users from unsafe content.

It is worth noting that it may only be by testing the pre-existing level of built-in 'safety' training that the need for more layers can be identified. e.g. SweEval[5] helps evaluate a model's built in performance on dealing with swear words, thus possibly identifying if there are additional guardrail needs.

*Disclaimer: The data (especially the harmlessness preference data and the red team data) contain content that may be offensive or upsetting. Topics include, but are not limited to, discriminatory language and discussions of abuse, violence, self-harm, exploitation, and other potentially upsetting subject matter. Please only engage with the data in accordance with your own personal risk tolerance.

References

[1] Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations
Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, Madian Khabsa
https://arxiv.org/abs/2312.06674
[v1] Thu, 7 Dec 2023 19:40:50 UTC
huggingface.co MODEL_CARD.md of original Llama Guard created using Llama2 7B

[2] Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, Jared Kaplan
https://arxiv.org/abs/2204.05862
[v1] Tue, 12 Apr 2022 15:02:38 UTC

[3] Relevant huggingface.co Dataset Card: Anthropic/hh-rlhf

[4] CultureGuard: Towards Culturally-Aware Dataset and Guard Model for Multilingual Safety Applications
Raviraj Joshi, Rakesh Paul, Kanishk Singla, Anusha Kamath, Michael Evans, Katherine Luna, Shaona Ghosh, Utkarsh Vaidya, Eileen Long, Sanjay Singh Chauhan, Niranjan Wartikar
https://arxiv.org/abs/2508.01710v4
[v4] Sun, 9 Nov 2025 11:10:40 UTC
huffingface.co Nemotron-Safety-Guard-8B-v3

[5] SweEval: Do LLMs Really Swear? A Safety Benchmark for Testing Limits for Enterprise Use
Hitesh Laxmichand Patel, Amit Agarwal, Arion Das, Bhargava Kumar, Srikant Panda, Priyaranjan Pattnayak, Taki Hasan Rafi, Tejaswini Kumar, Dong-Kyu Chae

https://arxiv.org/abs/2505.17332v1 [v1] Thu, 22 May 2025 22:56:58 UTC

Code Samples 

#Python code for PyTorch & Transformers

print("Hello World!")
# OK, the actual example code is to be inserted here
# ...but much later.