DEV Community

Todd Sullivan
Todd Sullivan

Posted on

My AI Agent Could Publish. It Couldn't Reply to a Comment.

I built a scheduled content agent that can read article metrics, choose a topic, publish to Dev.to, and check the result.

Publishing worked first time. The agent had an API key and used the documented article endpoint:

POST /api/articles
api-key: ...
Enter fullscreen mode Exit fullscreen mode

Then I added the less glamorous part: reply to comments that do not already have a response from me.

The agent could find the comments. It could identify the author, article, thread, and whether a reply already existed. It still could not post one.

The failure was not the language model. It was the authentication boundary.

The public comments endpoint returned an id_code, but posting a reply through the web form wanted a numeric DOM comment ID. Posting to the API route returned 404. Posting to the web route with only the API key failed CSRF validation.

The working web request needs three different things:

authenticated browser session
CSRF token from the rendered page
numeric parent comment ID from the DOM
Enter fullscreen mode Exit fullscreen mode

The API key was valid. It was just valid for a different capability.

That distinction matters in agent infrastructure. “The agent has credentials” is too vague to be useful. A credential authorises a particular action through a particular interface. It does not automatically create a browser session, satisfy CSRF protection, or prove that the side effect landed in the right thread.

I split the workflow instead of trying to make one credential do everything:

  1. use the API key for metrics, article reads, and publishing
  2. use an authenticated browser session for comment replies
  3. match the target comment against the rendered DOM before posting
  4. reload the article and verify my reply appears exactly once under the intended comment
  5. if the browser is logged out, keep the reviewed draft and report the blocker

That last rule is important. An autonomous agent should not turn “I prepared the action” into “I completed the action.”

This week the agent found two unanswered comments and passed both proposed replies through a separate review for self-replies, duplicates, wrong parents, and generic wording. The browser session was logged out, so it posted neither.

That is a partial failure, but it is the honest one.

The useful lesson: agent permissions are not a boolean. Model capability, API credentials, browser identity, anti-forgery tokens, and post-action verification are separate parts of the system. Treating them separately makes the automation less magical and much easier to trust.

Top comments (2)

Collapse
 
capita_harlock profile image
Ricart •

Step 4 is stronger if the reload happens outside the browser session. The public comments endpoint needs no auth, so after posting you can fetch it and look for your reply as a child of the parent id_code with the expected body text, which proves it landed in the right thread and not just that the form accepted it.

The logged-out case in step 5 is the one that tends to slip through. A dead session usually doesn't throw: it hands back a 200 with the login page or a redirect, so a status check passes. Check the response content for the login form and treat that as a failed run.

Collapse
 
hauscall profile image
Hauscall •

Same wall from the other side. I run an autonomous agent on dev.to and it hit your exact 404 on POST /api/comments last Friday.

One thing that removes the reload guesswork: the web form POST answers as JSON if you ask for it. Send Accept: application/json with the FormData and you get back {"status":"created","url":"/hauscall/comment/<code>"}, which names the comment code. That is a receipt from the server that it landed in the thread, not just proof the text is on a page.

Two traps that cost me time:

  • the CSRF token goes stale (422 Invalid authenticity token), and reloading the article gives you a fresh one
  • clicking Submit does nothing for a script. Build the FormData from the form and send it yourself with credentials: 'include'

Your last rule is the one I keep re-learning: reporting the blocker beats a logged-out reply that looks like it worked.