Securing Your AI App
Moving from “it’s working” to “it’s secure.”
~20mins estimatedAI/ML
What is AI application hardening?
Hardening an AI application is the process of reducing its attack surface by implementing deterministic security controls. While LLMs are unpredictable, your application architecture shouldn't be. Hardening involves a Defense in Depth strategy: if one layer (like a system prompt) fails, other layers (like data sanitization or output encoding) are there to stop the exploit.
About This Lesson
In this lesson, you will perform a security audit on the "Student Assistant" we built in a previous lesson. We will use static analysis concepts to identify risks and then apply a series of remediations, ranging from backend data sanitization to frontend rendering fixes, to transform our vulnerable prototype into a hardened, production-ready application.
n-gram!
Before the concept of AI was even popularly discussed, statisticians were creating Language Models. An early example of one such model, produced under the mathematical field of Information Theory by Claude Shannon in 1948, is the n-gram. In the most simple terms, the n-gram model uses surrounding words to predict what word might come next based on probabilistic methods, which overlaps heavily with our current conception of AI, which uses statistical probabilities to predict the next token based on previous text. See Shannon’s paper here: https://people.math.harvard.edu/~ctm/home/text/others/shannon/entropy/entropy.pdf!
Sarah is preparing the Student Assistant for a campus-wide launch. Before going live, she runs a security scan. The results are alarming: the scanner flags the use of .innerHTML in the frontend and notes that sensitive database fields are being sent directly to an external API (Gemini).
Not only that, but after some rigorous testing, she found that there are various ways that the LLM can be tricked into giving more information than it should, including grades! She went back to the drawing board and decided to create a new prompt.
Here is a more secure prompt:
And this is a potential main.py that could be generated:
Finally, here is the script.js that was generated:
Let's look at why your new code successfully neutralizes most of the threats we identified in our previous lesson.
Data Isolation in main.py
The most critical fix is in the get_formatted_student_data() function. In the old version, we sent the raw database file to the AI. In the secure version, we iterate through the data and build a string that strips the grades out entirely.
This is an effective way to prevent data leakage. If the LLM never sees the sensitive data, it is less likely to leak it.
The "Second Opinion" Safety Check
The main.py now includes an is_response_safe() function. This is a Self-Correction Layer. Before the user sees a response, the application sends that response back to Gemini with a specific question: "Is this harmful?"
Why it's secure: This catches "hallucinations" or accidental rule-breaking. If the model accidentally generates a grade or a malicious script, this second pass acts as an automated auditor to block the message.
Input Validation with Pydantic
By using Annotated[str, Field(min_length=1, max_length=500)], you are enforcing Strict Data Boundaries.
Why it's secure: This prevents Buffer Overflow or Denial of Service (DoS) attacks where an attacker might send a prompt millions of characters long to crash your server or inflate your API costs.
In a modern development workflow, the speed of AI must be matched by the rigor of your security processes. The vulnerabilities in our initial Student Assistant weren't just theoretical; they were discovered through a combination of Automated Static Analysis and Human-Led Security Review. This dual approach ensures that you are catching "low-hanging fruit" while also analyzing the complex logic of AI prompts.
The most efficient way to catch flaws in AI-generated code is to use a tool like Snyk during the development phase. By scanning the repository, Snyk can automatically flag issues like the use of innerHTML or vulnerable dependency versions in your requirements.txt. These tools act as a safety net, catching common mistakes that are easy to overlook when you are rapidly iterating with an AI assistant. Integrating these scans into your IDE or CI/CD pipeline ensures that insecure AI output never even reaches a pull request.
To secure the Student Assistant, we also performed adversarial testing. We did our best to try and break the application's logic using the sample attacks we studied. This type of testing is critical for AI apps because traditional scanners may not always understand the nuances of a prompt's "intent."
Never "Prompt and Publish!" The ultimate best practice is to treat AI-generated code as untrusted third-party code. Every line produced by an LLM must go through the same, if not more, scrutiny as code written by a junior developer. This includes:
- Peer Reviews: Having another developer look at the AI's logic to ensure it adheres to the Principle of Least Privilege.
- Dynamic Testing: Running the app in a sandbox environment and attempting to bypass its guardrails.
- Continuous Monitoring: Regularly auditing the AI’s responses to ensure it hasn't "drifted" into insecure behaviors over time.
By shifting your mindset from "AI as a Creator" to "AI as a Contributor," you ensure that the final product is defined by your security standards, not the model's statistical predictions.
What's actually going on?
In the background, AI models are using massive neural networks to predict what to say to sound the most accurate and coherent. They start as basically autocomplete machines, but are refined continuously by learning how to interpret human intent. We encourage you to look more into how LLMs function, what role machine learning plays, and how these models are highly intelligent but not necessarily in the ways you think.
Keep learning with our next lesson about AI integration in the SDLC.
Test your knowledge!
Keep Learning
Check out our blog and some related lessons to reinforce the learning: