AI cybersecurity: treat model HTML and SQL as untrusted

AI in a security product is a model that ranks or suggests. It is not a new authorization layer.

If you put an LLM in front of a ticket queue or a SIEM, you still need the same object checks, the same prompt-injection assumptions, and the same human who can say no. OWASP’s GenAI list is a reading order for those misses.

The usual mistake is treating a chatbot that can call tools as a trusted intern. The model will follow the untrusted text you gave it.

This page is where AI helps a defender, where it becomes another interpreter, and the OWASP items that map to code you already have.

OWASP dated the GenAI LLM Top 10 2026 as 4 August 2026. canonical 2026 README the same day LLM10:2026 Improper Output Handling. The miss it names is model text you pass downstream without encoding.A bio the model wrote, dropped into innerHTML, is the failure.

Keep the secure coding checklist next to this page. Read XSS when the sink is the browser, injection when the sink is SQL or the shell, and input validation when you still need an allowlist before either sink. This page will not walk a prompt-injection recipe. It will not sell an AI SOC.

LLM10 is a sink problem, not a paradox

The 2026 README lists ten entries. LLM01 is still Prompt Injection. LLM10 is Improper Output Handling. md. On the 2025 list, output handling was LLM05 and excessive agency was LLM06. The 2026 files renumbered both.

LLM10’s first paragraph says the bug is insufficient validation and handling of model output before it is passed downstream. The next sentences say that is not the same as LLM07 misinformation, and not the same as LLM01 input handling. Successful cases it names include XSS in the browser and SQL reaching a database. The mitigation list starts with a blunt line: treat the model as any other user.

That is the whole page. The model is a junior who can type HTML and SQL. Your renderer and your driver are the controls. A “paradox” framing is how the old copy avoided naming a sink.

LLM03 Excessive Agency is a different ticket: tools the model can call. This page does not scope agent permissions. If the model can only return text, LLM10 is the ticket. If it can call exec or a privileged API, stop and read LLM03_ExcessiveAgency.md. Do not merge those two into “AI risk.”

Model markup is still CWE-79

Firefox 148 shipped on 24 February 2026. The release notes and the Mozilla Hacks writeup name element.setHTML() next to Trusted Types. MDN’s setHTML page is the contract the XSS guide already cites: the method parses HTML and, with the default Sanitizer config, drops XSS-unsafe tags and handlers. renderModelHtml calls el.setHTML(modelHtml) with no extra config because that default is the safe one. It is not a text sink. Markup still becomes markup.

A model that returns a “safe bio” is still a string. innerHTML, dangerouslySetInnerHTML, v-html, and a Pug interpolation you marked unescaped all parse it. setHTML will strip a handler and still give you a heading in a username. If the field is a name, a title, a search term, or a status, the sink is textContent or the framework default escape. The XSS page is the longer sink list. This page only adds the source: the string came from a model.

function renderBio(el, modelText) {
 el.textContent = modelText;
}

renderBio is the named helper. It does not inspect tags. It does not call a sanitizer. It writes characters. The fallback when a browser has no setHTML and the product still wants HTML is not “paste it.” The fallback is still text until you have a reviewed policy on the XSS page.

function renderModelHtml(el, modelHtml) {
 if (typeof el.setHTML === "function") {
 el.setHTML(modelHtml);
 return;
 }
 el.textContent = modelHtml;
}

renderModelHtml is the named fallback for a CMS body you already decided must accept tags. Missing setHTML fails to text, not to innerHTML. Do not invent a regex that “strips script.” entuno’s sentence is why: sanitise is a marketing word. A history of holes is a first-party fact of HTML sanitizers, not a vibe.

// BAD: do not ship
// preview.innerHTML = await model.complete(prompt);

Markdown renderers are the other hatch. A chat UI that auto-fetches images in model Markdown is an outbound request the LLM10 file also names. Disable that fetch unless you proxy it. This page will not walk the exfil URL. Turn the auto-preview off. That is the control.

Model SQL is still CWE-89

LLM10 example 3 is the SQL case: queries the model wrote, executed without parameterization. I am not adding a payload. The injection page already has OpenEMR CVE-2026-24908 from 28 April 2026, a _sort token glued into ORDER BY. A model that emits ORDER BY is the same class with a nicer comment.

Interpolating a value into query text is templating, not a bind. The same split is on the database locks page, with the HN thread linked there. Your driver can send placeholders and values on separate channels. The model cannot. “Use parameters” in a system prompt is a wish. searchOrders is a control.

const STATUS = { open: "open", paid: "paid", void: "void" };

function searchOrders(db, orgId, statusToken) {
 const status = STATUS[statusToken];
 if (!status) {
 throw new Error("unknown status");
 }
 return db.query(
 "SELECT id, status FROM orders WHERE org_id = $1 AND status = $2",
 [orgId, status]
 );
}

searchOrders is the named helper. The model may suggest a status token. It never suggests a fragment. STATUS is the allowlist. A missing key throws. The SQL text is a literal you wrote. $1 and $2 travel apart from the values. That is the bind the injection page wants. An identifier you cannot bind, a column name, stays in a map you wrote, the same way OpenEMR should have mapped _sort.

// BAD: do not ship
// db.query(await model.complete("write SQL for " + req.query.q));

ORMs do not save a raw hatch. sequelize.query, knex.raw, Prisma $queryRawUnsafe, and TypeORM query() are the names that turn the stack back into a string. If the model filled that string, you are in LLM10 and CWE-89 at once. Grep those names. The injection page is the hatch table.

MODEL a bio, a status token, a "query"

HTML renderBio -> textContent
 characters on the page
 no node, no handler

SQL searchOrders -> $1 $2
 statement and values travel apart
 STATUS map holds the identifier

HATCH innerHTML = completion
 db.query(completion)
 the interpreter reads grammar

What a detector does not replace

no 2025 or 2026 first-party paper that measures “AI security products stop N percent of breaches,” so that number is not on this page. A detector that ranks log lines can be useful. It does not encode a bio. It does not bind a query. It does not review a PR.

Three claims I will not repeat:

• The model writes secure code by default. LLM10 scenario 6 is an app that compiles and deploys model code without review. Your CI still has to grep innerHTML and raw SQL. A green copilot badge is not a control.
• A prompt can replace a bind. The 2026 LLM01 file says models do not architecturally separate instructions from data. start of that file. A system prompt that says “never emit SQL” is not a prepared statement. searchOrders is.
• Sanitize means safe. entuno already refused that word. A library that claims to clean HTML has a CVE history. Encode at the sink. Bind at the driver.

If you bought a copilot, keep it. Use it to draft tests for renderBio and searchOrders. Do not let it write the sink. That is the honest split. It is not a paradox. It is a junior with a fast keyboard.

Review the string like a junior PR

A completion that lands in your repo is a pull request. A completion that lands in a response body is a user field. Both need a named function between the model and the interpreter. The checklist on this site is the rest of the floor. This gate is only the two sinks.

function completeSafe(prompt) {
 const text = String(prompt).replace(/[\u0000-\u001F\u007F]/g, "");
 return model.complete(text);
}

async function handleBio(req, res) {
 const hint = String(req.body.hint || "").slice(0, 200);
 const text = await completeSafe(hint);
 res.json({ bio: text });
}

completeSafe strips control characters and returns text. handleBio caps the hint at 200 characters. It does not return HTML. It does not return SQL. The JSON API sends a string. The browser calls renderBio. The search route never sees the completion. It sees statusToken from a form you wrote, then searchOrders. If a product manager wants “ask the model for the query,” the answer is no. Offer a token picker. Map the token.

Logging is the leftover hatch LLM10 names: ANSI and control characters in a terminal or a log pane. Strip those bytes before a viewer that interprets them. I am not walking OSC 52. I am telling you not to pipe raw completions into a pane that honors escapes. Print them as visible hex if you must keep them.

Grep the two sinks

You are not walking an exploit. You are proving the model never reaches innerHTML or a raw query. Run this on the app you own.

rg -n "innerHTML|dangerouslySetInnerHTML|v-html|bypassSecurityTrustHtml|queryRawUnsafe|sequelize\.query|knex\.raw|complete\(.*HTML" --glob '!node_modules' --glob '!dist'

A hit on innerHTML next to a completion is a review. A hit on $queryRawUnsafe next to a completion is a review. A hit on renderBio and searchOrders is the shape you want. Run the unit test under jsdom or Vitest’s happy-dom environment. Add a unit test that the bio node’s textContent equals the fixture string and that its innerHTML equals the escaped form of that string.

test("renderBio writes characters", () => {
 const el = document.createElement("p");
 renderBio(el, "<b>hi</b>");
 assert.equal(el.textContent, "<b>hi</b>");
 assert.equal(el.querySelector("b"), null);
});

That test is the five-minute proof. It does not need a live model. It needs the helper. If a teammate “just this once” assigned innerHTML, the test fails. Put it in the same job that runs the rest of the UI tests. A weekly detector that never saw this node is not a substitute.