Scraping Became unreliable

In the past few days scrapes that were working great have become unreliable. Has something changed with the Firecrawl connection?

Last time this happened it was because of that.

1 Like

Hi @Digidave, sorry to hear that, but thanks for flagging it.

Could you share a Loom and some screenshots of the debugger so we can compare the successful and failed runs? Also, are you scraping the same websites as before, or are there any new ones?

A link would also be helpful so we can better understand what might be causing the issue.

1 Like

I concur with DigiDave’s post. Firecrawl specifically is not working. I submitted a detailed email to Support on May 16. Firecrawl does not save output; even the screenshot variable is not saving output.

I noticed Firecrawl working inconsistency on May 15, but on May 16 it is not rendering any output. Alas, Firecrawl is required for the particular workflow I’m working on since I need the underlying HTML, which the Default scraper does not provide.

Firecrawl seems to have been having some issues over the past few days: https://firecrawl.betteruptime.com/

2 Likes

Hi Alex
Sorry - I will share a loom soon. Just been a bit swamped. But it does appear to be happening - relatively widespread. It’s not that it’s 100% broken. It’s sorta just unreliable in a way that it wasn’t before. I would say failure 50-70% of the time. Just coming up empty with a scrape.

I will point out what. Sean did above: It looks like firecrawl was having issues.

I only wonder if this is an issue that needs to be addressed via the API??? I don’t know: Scraping is down | Firecrawl