"Cannot retrieve data from the specified URL" error

cbaslancbaslan MemberPosts:6Contributor II
edited November 2018 inHelp

Hi,

I have a process containing a collection of example sets including URLs. I will loop through every example set using loop collection operand. Inside there is a get pages operand and store operand. Total number of websites to be crawled is about 20k from 10 different domains. But when started the process it gives this error and stops. But I know that I have connection and that page causing the error exists. What could be the problem here? Sceenshot is attached.

Thanks

12.png 61.7 k
Tagged:

Answers

  • IngoRMIngoRM Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University ProfessorPosts:1,751RM Founder

    Hard to say. Could be some Javascript or something to throw it off...

    In situations like this, where you loop through thousands of sources and might run into trouble with a few of them, I always recommend to use the operator "Handle Exception" as part or actually around the extraction. This way the rest of the process and the data collection can still finish even if one source throws the system off (or you are offline for 1 second or...)

    Hope this helps,

    Ingo

  • cbaslancbaslan MemberPosts:6Contributor II

    Thanks a lot! I will try that and see how it goes.

Sign InorRegisterto comment.