Yes! Facebook is now harder to detect my browsing behavior, they can’t know which line I stopped at and reading at it or how many times I scrolled on that page, etc.
I did the same thing again, go to Facebook -> Login -> Search the keyword -> Start scraping and now it works! This certainly reminded me of my high school time when smartphones were something rather modern and most of us were still relying on a compact phone. I started to think about what if Facebook doesn’t keep sending my browsing behavior? Will Facebook be unable to detect if I’m a bot or a human? So I went to my browser settings and disabled JavaScript to see how Facebook works within that constraint.įacebook suggested me to try the ‘mobile-optimized website’ which it is actually the old-school which doesn’t use any AJAX. Yeah, Facebook is good at detecting bots. Well, that quickly caught the attention of Facebook and denied access to more data from their server. What my Selenium did was: Go to -> Login -> Search the keyword -> Start scraping while scrolling down. If you are not familiar with this, Selenium is essentially a tool to automate your browser, allow you to control and use your browser as if a human is using it. Then, I tried to scrape the data by mimicking the behavior of a user using Selenium. What I did first is try to scrape but I quickly realize most data are fetched asynchronously using AJAX. The tool I used was Scrapy, a fairly comprehensive and easy-to-use data scraping library in Python. No Graph API access = No access at all!? Not exactly.įirst of all, I do not understand and I guess will never understand the rationale behind Facebook decision to take down the public info endpoint which you read it right! Public info, they are meant for the public audiences.
Despite the data being scraped are ‘publicly accessible’ data which are meant for public audiences, Facebook actually took down the Graph API endpoint on this particular resource. What bothered me was the access to Facebook data. There wasn’t lots of challenges in terms of technical. So, I evaluated his requirements, made sure that I was able to deliver, and took the assignment. Basically what he wanted was a process improvement on how his team works, an automated pipeline to scrape data from selected sources, process them and integrate them into existing system. Meanwhile, I was approached by a business owner asking me if I could help him out to create a scraping pipeline.
0 kommentar(er)
