![]() Note that findElement is different than findElements. Instead, you could use the more generic findElements function. So it seems to me that you should verify the existence of the element if possible, rather than trigger an exception. Is is better to anticipate and avoid exceptions, rather than handling them. You have this block (the nested try/catch block): try: # can't locate element - click the close on the popup add If find_all() can’t find anything, it returns an empty list. If you use BS, note the behavior of the different functions: You can either check with Selenium or BS4 if you have already retrieved the HTML. Regarding the scraping process, make sure that the DOM elements your are expecting are really present: websites change their layout more or less often and you must spot changes that you could break your application. More descriptive names would be job_title, job_posted_time etc
0 Comments
Leave a Reply. |