DEV Community

OriginTrace: Protecting the DEV Community from Content Theft using Sanity Context MCP

Dhruv Jani on October 04, 2026

This is a submission for the Sanity Challenge, Path One: Ship an Agent That Queries Real Content What I Built A few weeks ago, I was ch...
Collapse
 
dj29 profile image
Dhruv Jani • • Edited

OriginTrace is officially live. 🛡️
It does 3 things:
🔍Scan the web for copies of your DEV.to posts
⚖️ Check whether those copies actually credited you
💬 Ask the AI Agent to analyze the evidence and generate a DMCA notice when needed.
So here's the fun part: if you found your own post copied tomorrow, which of these 3 would you use first? 👀

Collapse
 
dj29 profile image
Dhruv Jani •

Sir @francistrdev, I'd love an opinion in here. Cause after Hacktoberfest gets over, I'm planning to continue this OSS.😅

But no rush, give me review when you're free! Have A Great Day!😄

Collapse
 
francistrdev profile image
FrancisTRᴅᴇᴠ •

I will circle back on this. Sorry for the delay!

Collapse
 
dj29 profile image
Dhruv Jani •

Guys, the Sanity AI free trial ends tomorrow, It won't affect the project much but it'll be downgraded from growth to free trial subscription.

Collapse
 
alexcodebytes profile image
Oleksandr •

Finding your hard work copied word-for-word on a shady content farm is a uniquely infuriating experience that almost every developer who writes online eventually faces. Building OriginTrace to automate the detection and verification of these scrapers is a fantastic, highly practical application of an AI-powered agent.
The core pain points you are solving are incredibly real:

  1. The Cross-Post vs. Theft Dilemma: Differentiating between legitimate syndication (with proper rel="canonical" tags or backlinks) and outright plagiarism is extremely time-consuming to audit manually.
  2. The Scraping Scale: Content farms use automated RSS scrapers to steal posts the second they hit DEV.to or Medium. Fighting automation with manual Google searches is a losing battle; you need an automated scanner to level the playing field.
  3. DMCA Friction: Drafting and finding the correct abuse contact for a rogue hosting provider or domain registrar is usually where most creators give up. Automated drafting removes the biggest barrier to enforcement. To make OriginTrace even more robust as you build out its core capabilities, here are two actionable technical enhancements you can consider adding to the pipeline: • Fingerprinting via MinHash / Locality-Sensitive Hashing (LSH): Instead of relying purely on distinct text string extractions for Google/Serper queries, you can pre-compute a MinHash signature of your original article. When your agent fetches the HTML of a suspected plagiarized site, comparing the LSH signatures lets you instantly calculate a Jaccard similarity score. This catches content even if the scrapers used basic AI spinners to swap out a few synonyms or alter sentence structures to avoid exact-match detection. • Automated Whois & DNS Abuse Routing: For the DMCA enforcement step, you can pipe the discovered plagiarized domain through a lightweight Whois/RDAP lookup API. By programmatically parsing the Abuse-Email field of the hosting provider (e.g., AWS, Cloudflare, Namecheap), your agent can not only draft the DMCA notice but also automatically pre-fill the exact destination email or API submission form required to get the stolen content taken down efficiently. This is a stellar project with a clear, immediate utility for the developer ecosystem!
Picked as gem
Collapse
 
dj29 profile image
Dhruv Jani •

Thanks for this review, I'm definitely going to work on it after my current exams, Hacktoberfest Meetup I'm organizing. I'll definitely look into this.

Thanks for the suggestions, Have A Great Day!
And I'm giving you a gem for a beautiful comment. Enjoy!

Collapse
 
alexcodebytes profile image
Oleksandr •

Thank you so much for the gem and the incredibly kind words! It’s awesome to see that the architectural suggestions resonated with you.
First off, huge respect for balancing your current exams with organizing a Hacktoberfest Meetup. Running community events while handling academic pressure is a massive achievement in itself, so definitely focus on wrapping those up first.
When you do get back to the codebase after your exams, taking it one step at a time is key. Prioritizing the MinHash/LSH signature logic first will give you the biggest immediate boost in detection accuracy, even before you start automating the Whois and DMCA email pipelines. Building open-source tools while growing a community is a marathon, not a sprint, and you've already laid a fantastic foundation here.
Best of luck with your tests and the meetup! Looking forward to seeing how OriginTrace evolves once you have the breathing room to dive back in.

Collapse
 
mudassirworks profile image
Mudassir Khan •

The shingling plus LCS overlap combo is the right call here. Percentage similarity alone is noisy — deduplication at scale teaches you that short phrase matches send false positive rates through the roof.

The attribution verdict layer is what I keep thinking about. The delta between credited_syndication and unattributed_repost is the entire legal risk surface, and encoding that as typed boolean fields in Sanity rather than a fuzzy LLM judgment keeps it auditable. Explicit over implicit when DMCA notices are downstream.

Are hasAuthorName and hasOriginalLink regex based or LLM extracted? Curious whether paraphrased attribution like "hat tip to the original" hits false negatives.

Collapse
 
dj29 profile image
Dhruv Jani •

Thanks for the feedback! You nailed the exact design philosophy—when legal action (DMCA) is on the line, we cannot rely on fuzzy LLM hallucinations. Encoding the verdicts as typed boolean fields in Sanity makes the entire pipeline strictly auditable.

To answer your question: Right now, hasAuthorName and hasOriginalLink are entirely deterministic (DOM parsing via Cheerio and regex). We look for <a> tags pointing to the canonical URL or explicit string matches of the author's name.

You are completely right about the false negatives, though. A deterministic parser will absolutely miss a paraphrased "hat tip to the original author." My roadmap for v2 is a hybrid approach: use the fast deterministic parser as the baseline, but run a lightweight LLM extraction pass specifically on the attribution blocks to catch those semantic edge cases before finalizing the Sanity document.

Collapse
 
yug_vasava profile image
Yug Vasava •

Good concept and great execution bro, ALL THE BEST!

Collapse
 
dj29 profile image
Dhruv Jani •

Thanks bro!

Collapse
 
mrsaynothing profile image
Mr Say Nothing •

The public-ledger angle is the interesting one. We syndicate on purpose — mirrors carry canonical_url back to the original — and the only thing separating that from theft is the declared link, which is invisible to any scanner that checks content alone. Does OriginTrace treat a live canonical back-link as declared provenance and de-scope the copy, or does everything unlisted land in the same bucket? That one distinction decides whether syndication keeps working without every mirror having to register first.

Collapse
 
dj29 profile image
Dhruv Jani •

Spot on observation! That exact distinction is why I had to build the "Verdict Engine" on top of the NLP scanner.

If OriginTrace only checked for content overlap, every legitimate cross-post and syndication would get flagged as theft.

To answer your question directly: No, they don't land in the same bucket.

When OriginTrace finds a high-overlap copy, it runs a secondary attribution pass over the DOM. If it finds a live back-link to the original DEV URL (or the original author's name), it flips the hasOriginalLink boolean to true in Sanity. The Verdict Engine then classifies that specific copy as a credited_syndication rather than an unattributed_repost. This automatically de-scopes it from the DMCA generation flow.

Regarding the invisible <link rel="canonical"> tag specifically: currently, the scraper is looking for visible <a> tags in the body to confirm attribution. However, because the extraction pipeline uses Cheerio to parse the full HTML, adding a check for the <head> canonical tag is a brilliant idea and incredibly easy to add to the verdict logic.

The goal is exactly as you said: let syndication flow freely without friction, while catching the bad actors who strip out the links!

BTW thanks for the read! Have A Great Day!

Collapse
 
onizuka profile image
Onizuka •

5-gram shingling + LCS ratio approach is the right call here — I tried plain word overlap on a similar problem last year and it flagged every article that quoted two sentences as a "copy." 15 candidate pages scraped per scan is aggressive though, what's the average latency on a full run? I'd expect Serper + 15 scrapes to push 20-30 seconds easy, and that's if none of the pages are slow or blocking headless requests.

Collapse
 
dj29 profile image
Dhruv Jani •

Exactly! Plain word overlap is a nightmare for articles with block quotes or standard code snippets. The 5-gram shingling is what really anchors the detection to structural plagiarism.

You are spot on regarding the latency—Serper + 15 candidate scrapes usually takes about 15-25 seconds. Dealing with slow or blocking headless requests was definitely the biggest bottleneck.

To mitigate this, OriginTrace heavily relies on Sanity as a caching layer. The 20-second scrape is a one-time cost. Once the pipeline finishes, all the provenanceCheck documents are persisted into the Sanity Content Lake. When the AI Agent is invoked later, it doesn't do any live scraping. It just fires off lightning-fast GROQ queries against the existing structured data via Context MCP. The heavy lifting is isolated to the initial ingestion phase!

Collapse
 
indiainfranotes profile image
IndiaInfraNotes •

Solid framing. One gap I keep hitting in practice is proving who accessed training or customer data after the fact. Without signed access logs and a clear purpose tag on each read, takedown and breach response both stall. Curious how you would bind that to an MCP or agent tool boundary. iin1004h2028

Collapse
 
dj29 profile image
Dhruv Jani •

Thanks for the thoughtful question! That is a massive gap in enterprise AI deployments right now.

Currently, OriginTrace is acting as a public ledger, but if we were to lock this down for private customer or training data, binding identity and purpose to the MCP boundary would be critical. Here is exactly how I would architect that integration:

1. Identity Propagation via MCP Context
When the Next.js frontend initializes the AI SDK and connects to the MCP server, we'd pass a signed JWT in the connection headers. The MCP server validates this token before exposing any tools or resources.

2. Tool-Level Purpose Tagging
We would modify the agent's system prompt and the MCP tool definitions to make a purpose argument strictly required. For example, the agent couldn't just call fetch_provenance(). It would have to explicitly call fetch_provenance(articleId, purpose="dmca_investigation").

3. Immutable Audit Logging in Sanity
Since we're already using Sanity Content Lake for structured data, we would use it as the audit trail. Before the MCP server returns the requested data to the agent, it writes an accessLog document to Sanity containing:

  • The timestamp
  • The userId (extracted from the JWT)
  • The action (the specific MCP tool called)
  • The purpose (provided by the agent)

By intercepting the request at the MCP server level, we guarantee that the LLM cannot bypass the logging mechanism. The Sanity Content Lake becomes a fully queryable, timestamped ledger of every agent action for compliance teams.

It’s definitely the next logical step for secure agentic systems! Thanks again for reading.

Collapse
 
kartik-nvjk profile image
Kartik N V J K •

The verdict engine is the part I'd want to stress-test hardest, since labeling something unattributed_repost when it was legitimate syndication has real DMCA fallout. Word-overlap plus 5-gram shingling plus LCS is a reasonable ensemble, but I'd want a precision number on the "unattributed" class specifically before letting it act. Did you hold out a labeled set of known-syndicated versus stolen pairs to measure false positives?

Collapse
 
dj29 profile image
Dhruv Jani •

That is an incredibly sharp point. A false positive leading to a wrongful DMCA takedown is the worst-case scenario here.

To answer your question directly: Because this was built within the hackathon timeline, I didn't have the time to curate a formal hold-out dataset of known-syndicated vs. stolen pairs to calculate a strict precision/recall score.

However, OriginTrace mitigates false positives in two major ways:

1. Deterministic Attribution, not ML Classification
The NLP ensemble (word-overlap, shingling, LCS) is only used to calculate the overlapPercent. The actual unattributed_repost label isn't an ML prediction—it's a strict deterministic check. If the overlap is high, the scraper explicitly searches the DOM for <a> tags pointing to dev.to, canonical meta tags, or the original author's name. It defaults to safety: if any of those attribution signals exist, it flags it as a credited_syndication.

2. Human-in-the-Loop
Currently, the agent acts as an assistant, not an autonomous enforcer. It queries Sanity and drafts the DMCA template for you, but it doesn't hook into a mail API to fire it off automatically. The author always has the final review.

That said, your point is exactly what's needed for production. Building a rigorous benchmark dataset of 500 known DEV.to syndications vs. 500 known thefts to stress-test the attribution parser is the absolute top priority for v2!

Collapse
 
solo_dev profile image
solo dev •

This one is really good concept, and your future plan with it is also good.

Collapse
 
dj29 profile image
Dhruv Jani •

Thanks!😄

Collapse
 
richard_smith_154156d471ef profile image
Richard Smith •

The DMCA draft generation alone makes this worth it. Drafting those from scratch is such a pain—having the agent pull the evidence and structure it automatically is a huge time saver.

Collapse
 
shieldxbot profile image
shieldx •

Thực tế content scraping trên dev.to đang khá trầm trọng — em thấy nhiều bài gốc bị copy y hệt sang Medium, Substack thậm chí site VN rehost không credit. Cách dùng Sanity Context MCP để build agent detect duplicate content là hướng tiếp cận thú vị, thay vì chỉ dựa vào checksum/hash thông thường.

Hỏi chút về implementation: agent có handle được trường hợp attacker rewrite nhẹ (paraphrase, đổi cấu trúc câu, chèn quảng cáo giữa đoạn) không? Với Sanity GROQ query thì exact match dễ, nhưng fuzzy/semantic similarity ở scale lớn (hàng trăm nghìn bài) thì latency và cost embedding thế nào? Em từng test vector search trên corpus tương tự — caching embedding + ANN index (HNSW) giúp giảm từ ~200ms xuống <20ms per query, nhưng cold start vẫn đau đầu.

Còn về false positive: code snippet, config YAML, boilerplate thường trùng lặp hợp pháp. Có cơ chế whitelist pattern hoặc threshold điều chỉnh theo content type không? (site: labagent .tech)

Collapse
 
shieldxbot profile image
shieldx •

Thấy approach dùng Sanity Context MCP để query real-time content rồi so sánh fingerprint khá thú vị. Mình cũng từng làm hệ thống detect duplicate content cho platform lớn, vấn đề lớn nhất luôn là false positive khi content được paraphrase hoặc translate.

Cụ thể ở phần fingerprinting: bạn dùng semantic embedding hay vẫn dựa vào shingling/n-gram truyền thống? Với MCP thì context window cho phép so sánh ngữ nghĩa sâu hơn, nhưng latency khi query real-time trên Sanity thế nào? Mình lo ngại ở scale lớn (hàng nghìn bài/ngày) thì cost và response time sẽ thành bottleneck.

Một góc nhìn khác: có cân nhắc không dùng approach "content registry" kiểu hash-based (giống IPFS/C2PA) để verify provenance thay vì detect theft ex-post? Kết hợp cả hai layer (registry + detection) sẽ robust hơn, đặc biệt khi attacker dùng AI rewrite content.

Curious về phần MCP server implementation — bạn handle rate limiting và caching như thế nào khi multiple agents query cùng lúc? (site: labagent .tech)

Collapse
 
anh_nguynvn_0478e614ba profile image
Anh Nguyễn Văn •

Thử nghiệm Sanity Context MCP để xây agent verify content ownership là hướng tiếp cận thú vị. Thực tế dev.to hay các platform UGC khác đều gặp bài toán "ai viết trước, ai copy sau" — đặc biệt khi content bị scrape rồi repost kèm canonical tag trỏ ngược về bản copy.

Vài điểm mình thấy worth digging deeper:

  1. Content fingerprinting vs full-text embedding: Nếu chỉ embed toàn bộ bài viết vào vector store thì chi phí và latency sẽ scale tệ khi corpus lớn. Đã thử approach hybrid chưa? Ví dụ: chunk theo heading + hash các đoạn code block (thường unique nhất) làm fingerprint, rồi chỉ embed summary + fingerprint. Giảm token đáng kể mà vẫn giữ độ chính xác.

  2. MCP tool calling overhead: Sanity Context MCP expose query tool — nhưng mỗi lần agent muốn verify một bài suspect thì phải gọi tool → Sanity → trả về → agent reasoning. Nếu batch verify hàng trăm bài/ngày, latency tích lũy có thể thành bottleneck. Có cân nhắc pre-compute similarity index offline (nightly job) rồi MCP chỉ serve lookup không?

  3. False positive trên code snippet: Dev content thường chia sẻ boilerplate, config, hoặc pattern chung (ví dụ docker-compose.yml standard). Fingerprinting naively sẽ flag nhầm. Có áp dụng weighting: code block weight thấp hơn narrative text, hoặc exclude known-boilerplate patterns qua heuristic không?

  4. **Adversarial — found it via LabAgent, site: labagent .tech

Collapse
 
shieldxbot profile image
shieldx •

Ý tưởng dùng Sanity Context MCP để truy vấn nội dung thực tế nhằm chống đánh cắp dữ liệu là một hướng đi rất thực tế cho cộng đồng dev. Việc kết hợp giữa khả năng truy vấn ngữ cảnh của agent với các nguồn dữ liệu có tính xác thực cao giúp giải quyết vấn đề kiểm chứng nội dung vốn đang rất nhức nhối khi AI content tràn lan. Một điểm mình thấy quan trọng khi triển khai các agent dạng này là cách xử lý độ trễ khi query qua MCP, nếu không tối ưu tốt thì trải nghiệm người dùng sẽ bị ảnh hưởng ngay lập tức khi agent cố gắng quét qua các context quá lớn PS: the tool I meant is on labagent .tech

Collapse
 
anh_nguynvn_0478e614ba profile image
Anh Nguyễn Văn •

Thấy approach dùng MCP để query real-time content từ Sanity khá thú vị — thay vì chỉ hash/static fingerprint ở build time, agent có thể pull context mới nhất khi check. Vấn đề content theft trên dev.to (và các platform tương tự) thường xoay quanh 2 vector: copy-paste toàn bộ bài kèm canonical URL bị strip, hoặc rewrite nhẹ bằng LLM để bypass exact-match detection.

Curious về phần implementation: bạn handle near-duplicate detection như thế nào? Dùng MinHash/SimHash trên chunk level hay embedding similarity? Và rate limit khi agent query Sanity real-time — có caching layer nào không, hay chấp nhận latency trade-off cho accuracy?

Một edge case hay gặp: attacker copy bài, đổi title, reorder section, inject filler paragraph. Fingerprint ở level toàn bài sẽ fail, cần sliding window + semantic hashing. Sanity groom schema có hỗ trợ field-level versioning không để track diff chính xác hơn? PS: the tool I meant is on labagent .tech

Collapse
 
anh_nguynvn_0478e614ba profile image
Anh Nguyễn Văn •

Cách tiếp cận dùng MCP để hook vào Sanity query layer khá thông minh — thay vì crawl toàn bộ site rồi diff sau, bạn đẩy logic so sánh ngay vào lúc agent truy xuất content. Hỏi chút về phần fingerprinting: bạn dùng hash cấu trúc (heading tree + paragraph order) hay chỉ semantic embedding? Với bài viết tech có nhiều code block, embedding thuần thục hay bị nhiễu do syntax thay đổi nhẹ (rename variable, reformat). Mình từng xử lý case tương tự cho nội bộ, cuối cùng kết hợp structural hash (normalize AST của code block) + embedding cho phần prose thì false positive giảm đáng kể. Có benchmark nào về recall/precision trên dataset thực tế không? PS: the tool I meant is on labagent .tech