How to save all information from Facebook profiles
In this article, we will talk about the Ultimate Facebook Scraper (UFS) program. It saves photos, videos, messages, lists of friends, colleagues, etc., events and all other information from the Facebook user profile. You can specify a list of a large number of users and UFS will save their information in a consistent and structured way.
For the program to work, you need to specify the email and password of the Facebook account. This program cannot retrieve data that is not accessible to the user, which credentials you use. That is, for example, if friends lists are visible only to friends, and your current user is not a friend, then the friends list will not be retrieved.
Another important note: Facebook often changes the names of div tags in the HTML source code, so the tool may work better or worse, depending on the changes made in the social network and in the UFS code. By the way, if you liked this program, you often use it and your skills are enough to find changes and fix the source code of Ultimate Facebook Scraper, then you can join the community and help keeping the tool updated.
By the time of writing, by the way, photos and videos are not saved – I hope that they will fix it soon. Or you can do it yourself. By the way, the program is an interesting example of how you can automate actions and parse data from a site that has good protection from bots. The approach used in Ultimate Facebook Scraper allows you to bypass almost any protection from bots.
For the program to work, you definitely need a graphical environment, since it will launch a full Google Chrome browser and will perform all actions in it. The program can work on Linux and Windows, but on Windows you will first need to configure the ability to run Python scripts.
For the program to work, you need to make sure that the program can start Google Chrome with the privileges of a regular user. The easiest way to implement this is to create a new Kali Linux user and log in under him.
How to install Ultimate Facebook Scraper (UFS) on Kali Linux
Create a new user in Kali Linux:
useradd -m -G sudo -s /bin/bash new_user
Replace new_user with the desired name.
Set a password for it:
Install Google Chrome on Kali Linux, for this:
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb dpkg -i google-chrome-stable*.deb apt-get install -f
Verify that the program can launch:
xhost + && su -c /usr/bin/google-chrome new_user && xhost -
Now reboot and log in as a normal user (not root).
Install the required dependency (required without sudo!):
pip3 install selenium
Download the program itself:
git clone https://github.com/harismuneer/Ultimate-Facebook-Scraper cd Ultimate-Facebook-Scraper/src/
wget https://chromedriver.storage.googleapis.com/`curl -s https://chromedriver.storage.googleapis.com/ | grep -E -o '<Key>[0-9.]+/chromedriver_linux64.zip' | tail -1 | sed 's/<Key>//'` rm chromedriver unzip chromedriver_linux64.zip
How to run Ultimate Facebook Scraper on Kali Linux
To credentials.txt file
bring the email and password of your existing Facebook account:
email = "" password = ""
In the input.txt file, enter the list of target profiles:
As targets, you need to specify URL addresses on Facebook profiles – one address per line, for example:
In scraper.py file
you can make some settings:
# whether to download photos or not download_uploaded_photos = True download_friends_photos = True # whether to download the full image or its thumbnail (small size) # if small size is True then it will be very quick else if its false then it will open each photo to download it # and it will take much more time friends_small_size = True photos_small_size = True total_scrolls = 2500 current_scrolls = 0 scroll_time = 8
During operation, the program will open a web browser, enter a username and password, go to the profile page of your interest and will walk through sections and save data.
The collected data is saved in the Data folder, separately for each profile.
The program does not work perfectly, as already mentioned, the program does not currently save photos and videos. In addition, sometimes the program may stop spontaneously, or errors from the web browser and driver appear.
Consider a few more problems and solutions.
Kindly replace the Chrome Web Driver with the latest one from
Full error output:
Starting Scraping... Kindly replace the Chrome Web Driver with the latest one from http://chromedriver.chromium.org/downloads and also make sure you have the latest Chrome Browser version. Your OS: linux
- Google Chrome is not installed
- too old version of Chrome Web Driver (this file comes with UFS itself)
- you are running as root and therefore Google Chrome cannot start
- different versions of Google Chrome and Chrome Web Driver (in this case, go to http://chromedriver.chromium.org/downloads and download the version you need)
To check the version of Chrome Web Driver:
You can see the version of the Google Chrome browser in the ‘About’ menu.
Your email or password is missing. Kindly write them in credentials.txt
You need to fill in the credentials in the credentials.txt file.
Input file is empty.
You need to add one or more targets (links to profiles on social networks) in the input.txt file.
- Open source research with OSRFramework (search by mail, nickname, domain) (100%)
- How to see locked HTML code, how to bypass social content lockers and other website info gathering countermeasures (100%)
- How to install ꓘamerka (kamerka) (100%)
- Utilities for information gathering, OSINT and network analysis in Windows and Linux (100%)
- How to find usernames on social media using Sherlock (100%)
- Guide to GPS Metadata in Photos (Part 2): How to understand, extract and convert geographic coordinates (RANDOM - 50%)