Recently we analyzed the architecture of the MemPalace agent, which promised to revolutionize RAG, achieved impressive metrics, and boasted an innovative "Memory Palace" from a Hollywood actress.

Well, independent reviewers and the community have now cracked open the source code. Spoiler: miracles don't happen, and open-source marketing has finally mutated into info-gypsy charlatanism 🤡

Here's how "star" AI projects are made nowadays.

🕵️‍♂️ Drama Behind the Scenes
As rightly uncovered on X, the project's co-author Ben Sigman is a crypto founder whose repositories are 80% Bitcoin scripts. The MemPalace code and benchmarks were written by a hired developer, Lu, who is neither listed as a co-author nor in the README. The entire git history was squashed into a single commit and rewritten under Milla Jovovich's account (who has 2 active days on GitHub ever).

But marketing is one thing. What about the actual results?

The Gap Between README and Reality
Experts in agentic memory conducted a full audit and filed an issue, revealing that the project's documentation is a collection of fairy tales:

1️⃣ "96.6% on LongMemEval"
This was presented as the achievement of their "Memory Palace." In reality, this score was obtained in raw mode—i.e., by searching through raw, uncompressed text "out of the box" in ChromaDB. The architecture of rooms and halls was not even used. They simply measured default ChromaDB embeddings and passed them off as their innovation.

2️⃣ "30x Lossless Compression" (AAAK)
This is not compression at all. It's a shoddy regex script that simply truncates sentences to 55 characters. The original text cannot be reconstructed from this (lossy algorithm). And guess how they calculated tokens to claim 30x compression? Through a hardcoded len(text) // 3.
By the way, when this "compression" is enabled, the benchmark score drops from 96.6% to 84.2%.

3️⃣ "Automatic Contradiction Detection"
In the knowledge_graph.py code, it simply does not exist. Facts are just dumped into a local SQLite database.

4️⃣ "+34% Boost from Palace Architecture"
This is ordinary metadata filtering (when we narrow the search to a specific wing or room). A standard feature of any vector database since time immemorial, not a breakthrough in RAG.

🏳️ Repentance
When the heat was on, the authors pushed an update to the repository. "Sorry, we miscalculated tokens, the compression is actually lossy, contradiction detection is not yet connected, and we slightly embellished the metrics."
"Thank you for the brutal and honest criticism," they wrote. No kidding, when the community caught you red-handed manipulating benchmarks.

Wrapping standard ChromaDB library methods in a beautiful "Mind Palace" metaphor, seasoning it with a celebrity face, and claiming a revolution—that's what AI hype can do.