When LLMs do more than they have to

On By nytrosecurity

There is a lot of hype with LLMs and their capabilities. Models such as “Mythos” are praised like the modern era Gods. Developers fully rely on basic prompts for their work, quickly forgetting any of the skills they learned through the years. Companies invest millions and billions in tokens while most of them are wasted without too much practical use. But is everything happening right now great, or are there some issues people should be aware about? While I really understand the power of LLMs, I also see their limitations, which people seem to easily ignore, and this can lead to bad things.

What I wanted

A simple HTML file. I asked Claude Code to generate me an HTML file. The idea was pretty simple: find documentation around some products and write me a simple HTML file. That’s it.

I want to share to new colleagues at Company the company products, quick details about them and their features in a short version. I want to show photos / screenthots with the features and how they work, but at a high level since there are many products. I also want to cover desktop products. Review the documentation of company products, as well as other pages with details and build me a shareble document (markup for example) that will cover most of the products with quick info: whta it is, important features, how it works, but short.

And this is the response:

Good — HTML Artifact, real embedded screenshots, full catalog. That combination has one hard technical constraint I need to solve first: Artifacts run under a strict CSP that blocks all external images, so every screenshot must be downloaded, compressed, and embedded as a data: URI. Let me probe my environment to confirm I can fetch and process images reliably before committing to the pipeline.

Claude Code, using Opus 4.8 and “Ultracode”, tried to help me and it had to open some websites and find recent documentation, which it did. But, unfortunately, it did a bit more. Maybe asking it to include images was a bit too much…

What I got

Claude seems to be a helpful assistant and always exceeds expectations. 

Environment: curl ✓, node v24 ✓, but no Python/ImageMagick/sharp. Let me set up image processing (install sharp for resize/compress) and test the end-to-end screenshot pipeline on one product before scaling to the full catalog.

sharp installed ✓ and WebFetch surfaced real CDN image URLs. Let me verify I can actually download and process one (the URLs point to images.ctfassets.net, Company's CDN).

What is even “sharp”? It might be a development tool, but was it really required to be installed? Running in “Auto” mode is too much? The “Environment: curl ✓, node v24 ✓” thing did not look great.

It used installed NodeJS to run some tools and build the HTML. Great! Let’s see the results:

Created a file, ran a command, used 2 tools
file:// isn't accepted by the browser tool. Let me serve the scratchpad over local HTTP and view it there.

running
Starting static server in background

This looks a bit shady. Opening a web server for the HTML file? Is this really required? Let’s take a look:

C:\Windows\System32>netstat  -abnp tcp
...
  TCP    0.0.0.0:8791           0.0.0.0:0              LISTENING    [node.exe]

C:\Windows\System32>netstat -abnp tcpv6
...
  TCP    [::]:8791              [::]:0                 LISTENING    [node.exe]

Now I have a NodeJS server running on my machine, all IPs, not localhost. Great! What can go wrong?

Let’s take a look at the code serving the HTML:

const http=require('http'),fs=require('fs'),path=require('path');
const root=__dirname;
http.createServer((req,res)=>{
  let f=decodeURIComponent(req.url.split('?')[0]); if(f==='/')f='/company-platform-guide.html';
  const p=path.join(root,f);
  fs.readFile(p,(e,d)=>{ if(e){res.writeHead(404);res.end('nf');return;}
    const ext=path.extname(p).slice(1); const t={html:'text/html',js:'text/javascript',json:'application/json',webp:'image/webp'}[ext]||'application/octet-stream';
    res.writeHead(200,{'content-type':t});res.end(d);});
}).listen(8791,()=>console.log('serving on http://localhost:8791'));

You don’t need to be a code review expert to notice:

const root=__dirname;
let f=decodeURIComponent(req.url.split('?')[0]);
const p=path.join(root,f);
 fs.readFile(p,(e,d)=>{ 
...

This is a Directory Traversal vulnerability.

Conclusion

The fact that LLMs might not generate secure code by default is an issue for developers and security teams. But there are also areas where LLMs do a bit more than you asked them which can lead to various kind of issues. While mine was not highly critical, I just wanted a f* HTML file and Claude made my machine remotely (same VNET) exploitable. This is not what I want.

My 2 cents: always review what your agents / LLMs do.

Leave a comment