I want to build some script which will include site cloner, so i wonder if anyone here have experience with that? Which libraries were used or framerwork, i need information. Cheers
Forum Thread: Anyone Have Experience in Cloning Site via Python?
- Hot
- Active
-
Forum Thread: When My Kali Linux Finishes Installing (It Is Ready to Boot), and When I Try to Boot It All I Get Is a Black Screen. 8 Replies
5 days ago -
Forum Thread: HACK ANDROID with KALI USING PORT FORWARDING(portmap.io) 12 Replies
2 wks ago -
Forum Thread: Hydra Syntax Issue Stops After 16 Attempts 2 Replies
1 mo ago -
Forum Thread: Hack Instagram Account Using BruteForce 208 Replies
1 mo ago -
Forum Thread: Metasploit reverse_tcp Handler Problem 47 Replies
2 mo ago -
Forum Thread: How to Train to Be an IT Security Professional (Ethical Hacker) 22 Replies
3 mo ago -
Metasploit Error: Handler Failed to Bind 41 Replies
3 mo ago -
Forum Thread: How to Hack Android Phone Using Same Wifi 21 Replies
3 mo ago -
How to: HACK Android Device with TermuX on Android | Part #1 - Over the Internet [Ultimate Guide] 177 Replies
3 mo ago -
How to: Crack Instagram Passwords Using Instainsane 36 Replies
3 mo ago -
Forum Thread: How to Hack an Android Device Remotely, to Gain Acces to Gmail, Facebook, Twitter and More 5 Replies
3 mo ago -
Forum Thread: How Many Hackers Have Played Watch_Dogs Game Before? 13 Replies
3 mo ago -
Forum Thread: How to Hack an Android Device with Only a Ip Adress 55 Replies
4 mo ago -
How to: Sign the APK File with Embedded Payload (The Ultimate Guide) 10 Replies
5 mo ago -
Forum Thread: How to Run and Install Kali Linux on a Chromebook 18 Replies
5 mo ago -
Forum Thread: How to Find Admin Panel Page of a Website? 13 Replies
6 mo ago -
Forum Thread: can i run kali lenux in windows 10 without reboting my computer 4 Replies
6 mo ago -
Forum Thread: How to Hack School Website 11 Replies
6 mo ago -
Forum Thread: Make a Phishing Page for Harvesting Credentials Yourself 8 Replies
6 mo ago -
Forum Thread: Creating an Completely Undetectable Executable in Under 15 Minutes! 38 Replies
8 mo ago
-
How To: Crack SSH Private Key Passwords with John the Ripper
-
How To: Scan for Vulnerabilities on Any Website Using Nikto
-
How To: Crack Password-Protected Microsoft Office Files, Including Word Docs & Excel Spreadsheets
-
How To: Hack Apache Tomcat via Malicious WAR File Upload
-
How To: Dox Anyone
-
How To: Use Burp & FoxyProxy to Easily Switch Between Proxy Settings
-
How To: Get Root with Metasploit's Local Exploit Suggester
-
How To: Find Identifying Information from a Phone Number Using OSINT Tools
-
BT Recon: How to Snoop on Bluetooth Devices Using Kali Linux
-
How To: Enumerate SMB with Enum4linux & Smbclient
-
How To: Brute-Force FTP Credentials & Get Server Access
-
How To: Find Passwords in Exposed Log Files with Google Dorks
-
Tutorial: Create Wordlists with Crunch
-
How To: Target Bluetooth Devices with Bettercap
-
How To: Use Ettercap to Intercept Passwords with ARP Spoofing
-
How To: Create a USB Mouse Jiggler to Keep a Target Computer from Falling Asleep (& Prank Friends Too)
-
Hack Like a Pro: Finding Potential SUID/SGID Vulnerabilities on Linux & Unix Systems
-
How To: SQL Injection Finding Vulnerable Websites..
-
How To: Perform Local Privilege Escalation Using a Linux Kernel Exploit
-
How To: Change a Phone's Coordinates by Spoofing Wi-Fi Geolocation Hotspots
2 Responses
That's actually a research project I'm working on, and you can check it out here:
https://github.com/AlexMapley/Bartimeaus/blob/master/spider.py
The series I'm writing right now is actually a build up to this point.
https://null-byte.wonderhowto.com/forum/creating-python-web-crawler-part-1-getting-sites-source-code-0175912/
Although you'd definitely have to tweak this program a little bit, it's designed to go through an entire website and archive all of it's pages. You could definitely use it to clone a website.
To run it from the terminal, run "python spider.py 'http://www.example.com 1"
What it will do is start from the website link you input as argument 1, and archive every single linked webpage with the keyword "example". It will call itself recursively 1 time, or however many times you put in argument 2, opening every link it sees from every page. It will also never open the same link twice.
If you run this on a website, it'll probably take a while (maybe an hour???) but you can definitely clone it.
Thank you, that was very helpful !
Share Your Thoughts