Live Crawling doesn't work at all

What solution did you choose ?

A post was merged into an existing topic: Live Crawled URL Prompt is not working

The issue is usually related to the local network or VPN. Trying a different network, VPN, IP address, or device can help.

I continue to receive the error message from Chat GPT when using live crawling of “live crawling of “xx” failed. Please try again.” then GPT responds that it cannot browse the internet. Message attached.
Failed crawling

Trolled forums for posts that are related to this topic and tried, linking GPT-4 email and AIPRM emails, Checked VPN and web browser, website does not have code to stop scraper/crawlers, Tried GPT-4 Browsing, plug-in and other models.

Please help, only reason to upgrade to Elite.

Sorry to hear that you are having trouble with the Live Crawling of URLs. I have tested the Live Crawling feature using the same URL and prompt template visible in your screenshot and have not found any issues yet. It appears to be functioning correctly.

See also:

Hi, The Live Crawling function of outranking an article is nog longer working. I get a grey box at the top mentioning: “Live Crawling of “https://URL/” finished - parsing content …”

and then ChatGPT answers:
It looks like you’ve provided a URL from “URL” that seems to be related to [topic]. However, I’m unable to access external websites or specific content on the internet in real-time. If you have any questions or need information related to [topic], feel free to ask, and I’ll do my best to assist you based on the information I have up to my last update in September 2021.

What am I doing wrong?

Thanks in advance!

Hi Marcel, I’m sorry to hear that the Live Crawling of URLs didn’t work for you. Please refer to the information provided here:

doesnt work for live crawling but its ok for webpilot plugin, why?

Apologies for any inconvenience caused. Based on the attached screenshot, it appears that the parsing of content after live crawling of URL was not successful. This issue seems to be related to parsing of Notion pages that require JavaScript.

it is not working today - i tried many time - please check!

The Live Crawling feature appears to still be functioning properly. However, some websites might block live crawling or any kind of crawlers, which could be the reason you are unable to use this feature on specific sites.

1 Like

Hi. for 3 days now aiprm’s live crawling feature is not working. this is very important for me. how can i fix this problem?

Read above post

We’re unable to run a certain prompt. It says that live crawling is failed. Please see the screenshot.

1 Like

@Rome_and_Byzantium it seems like the Live Crawling of is functioning properly.

Can you please check it again?

If it doesn’t work can you please share the prompt as well?

I created my own prompt for crawling LinkedIn profiles.
Sometimes it is working, but sometimes I get notification that because of LinkedIn rules I cant crawl it.
Did anyone had similar problem? What might be a solution?

:arrow_up: Please refer here:

Thank you for reply, I know that it might work at your place, but at mine still is not.
Are there any specifics regarding the LinkedIn crawling?
Is there your like live support, or meeting support or similar for Elite plan users?

Note that some websites like LinkedIn might block live crawling or other crawlers which might be the reason why it is not working on your end.

At this time, we only provide support through emails. If you encounter any issues or have any other questions, you may contact us at using the email address that is linked to your AIPRM subscription.

just to tell you this is still happening and it always goes like this:

  1. I enter one linkedin profile and it does its work
  2. I enter second one, and I got this message:
    I’m unable to access LinkedIn profiles directly. If you’re looking for specific information about Sofia’s professional background, skills, or experience, I recommend visiting her LinkedIn profile for the most accurate and up-to-date details. LinkedIn profiles are a great resource for professional information, including work history, educational background, certifications, and endorsements from colleagues and peers.
  3. I close my browser, open again, and start from the begining

You can try reselecting the live crawling prompt you used using the AIPRM sidebar: