How Developers Joyfully Shoot Themselves in the Foot 🔫

On crackr.dev there is a catalog tracking vibe coding failures.

Here are a couple of examples related to Python:

1️⃣ Supply chain via hallucinations
Top LLMs consistently advised installing a non-existent package huggingface-cli (in reality it's installed via pip install "huggingface_hub[cli]").
The folks at Lasso Security didn't miss a beat and uploaded a dummy package with that name to PyPI.

The result — over 30,000 real downloads in a couple of months. That's 30 thousand developers who mindlessly copy-pasted a command from a chat into their terminal. An ideal attack vector: you don't even need to hack anyone, just wait for the AI to feed your malware to the victim.

2️⃣ RCE out of the box from Claude
The Red Team at Databricks asked Claude to whip up a multiplayer snake game. The model produced fully working code, clients connect, snakes slither. But to transmit state over the network, Claude decided to use pickle instead of JSON, because it's easier to serialize Python objects.

For those who forgot the basics: pickle.loads() on raw data from a socket is instant Remote Code Execution. Any connected client could send specially crafted bytecode and execute arbitrary commands on the server. Saved time on JSON parsing — got a hole the size of a hangar.

The problem with neural networks isn't that they're stupid. They write code that solves the task and works. But architectural security and edge cases are not considered by the network unless explicitly forced to do so through paranoid prompts.

Review, review, review 🫵