Setting Up an Environment for Web Hacking

A Note on Operating System (OS) / Distros

One of the first things people get caught up on when they start hacking is "which distro should I use?" Most, if not all, hacking tools work on any Linux / Unix operating system and are written in high-level, cross-platform languages like Python, Java, Golang, and Ruby, which will work on any OS. 

For the purpose of this article, using a Debian-based Linux distribution such as Kali, Ubuntu, or ParrotOS will make the setup more convenient; otherwise, don't get caught up on distro choice, just get started with whatever is most convenient for you. It won't make much of a difference as long as you start hacking.

For security purposes, you may wish to set up a Virtual Machine (VM) for web hacking locally. A VM will ensure that your host machine remains secure, and you can replicate your hacking environment easily by cloning a VM. You can follow the Kali Linux setup documentation for VirtualBox, VMWare, and UTM (on Mac) for VM setup.

Local vs. Cloud environments

Your hacking environment can be set up locally, or in the cloud using a VPS (Virtual Private Server). A VPS is a server that you rent in the cloud, with any OS you want, root access, and very fast internet speed. You should run some tools locally for faster interactivity, such as those with a GUI that you often interact with; others, such as scanning and enumeration tools, might be faster on a VPS (since the data centers they sit in have much faster internet), so that you can leave them running overnight or days on end in something like a tmux window. Having a VPS also means you have a reachable server on the internet to receive callbacks such as reverse shells and out-of-band HTTP and DNS requests. Setting up a VPS is not expensive; you can rent one for about $10 per month. We recommend a hybrid approach where you have all of the tools available in a local VM, with a VPS server available to install and run tools anytime.

Methodology Matters

At the end of the day, methodology, skill, and experience matter more than the tooling. This article tries to introduce tools that would be useful to different hacking methodologies and introduce you to ways of finding bugs that have good coverage.

When you're just starting out, tools guide you and teach you; but as you get more comfortable with hacking, you will be the one customizing and guiding the tools to achieve creativity in your own testing. This article won't go too deep into the advanced usage of each tool, but just enough to get you started.

The hacking environment can also include extra resources that aren't so much hacking tools but help you manage tasks, keep track of notes, and give you ideas when you're stuck, such as note-taking apps, methodology handbooks, and payload cheatsheets. Those "soft resources'' can make all the difference to keep you going.

Note-taking and Text Editors

Good note-taking is essential for tracking progress when hacking a web application. One of the most popular ways to take notes while hacking is to use markdown note-taking apps, which provide a simple syntax for highlighting, and denoting code snippets and TODO items.

There are many popular markdown-based note-taking apps, such as Obsidian, Notable, and Joplin. Pick one that works for you and stick with it - jumping around could be distracting and lead to data loss.

Heynote is a different kind of note app that's more of a "scratch pad" with one continuous buffer with syntax highlighting; you can add new snippets to it and use it as a space to store and edit payloads.

You may also wish to have a text editor like Sublime Text or VSCode for writing scripts and reading source code that you get along the way.

Having pen, paper, and sticky notes around the desk can also help those that are more inclined to physical note taking and drawing mind maps. 

Handbooks And Cheatsheets

There will always be times when you are testing a web app, and you get stuck, run out of ideas, or don't know what to do next. That's why there are some handy resources that you should keep locally as a reference guide. Simply download the PDFs, copy them into the Documents folder for safekeeping, and add the URLs into your bookmarks:

• Hack tricks 
• Payload All The Things 
• OWASP WSTG / Web Security Testing Guide: PDF or web version

Web Request Interceptors

Let's get into the "hard" tools themselves. A way to inspect, intercept, and manipulate HTTP requests with a proxy is essential to web hacking. The three most popular tools for web request interception are Burp Suite, Caido, and OWASP ZAP; the de facto is usually Burp. Good news - when new hackers reach at least a 500 reputation on HackerOne and have a positive signal, they are eligible for 3 months free of Burp Suite Professional.

Setting Up Burp

If you're using Kali, Burp Suite Community version is already installed by default. If it isn't, you can install it by running:

apt update; apt install burpsuite

(run any "apt" related commands as root by adding "sudo" in front of it or by having a root shell)

On other systems, I recommend downloading Burp from PortSwigger's official website to get the latest version. On Linux, it will download as a .sh file that you can run to install it.

Once it's installed, open up Burp, fire up a new project, and go to the "Target" tab; there, you will find an "Open browser" button that already has the proxy configured to forward traffic to Burp. Now you can start browsing to target websites and seeing the traffic come in!

You may also wish to go into Burp -> Settings and search for the word "dark," then enable dark mode in the display settings:

Setting Up Burp Extensions: Autorize

There's a large number of extensions available in Burp built by the community. Portswigger even has a blog on the best Burp extensions. Here we will cover how to set up Autorize, one of the best extensions that helps you find authentication (authn) and authorization (authz) related vulnerabilities such as IDORs and broken access control bugs.

Go to the Extensions tab, and then open the BApp Store. Here you will find a list of extensions available to be installed. Search "autorize", and click on the entry.

Since Autorize is an extension written in Python, you will need to download Jython and configure Burp to use it in order to run Python extensions. Click on "Download Jython" and download the standalone version.

Now you must configure Burp to use the Jython jar file we downloaded. Go into Burp -> Settings again, and search Jython. In the Extensions area, we can choose "Select file" next to "Location of Jython standalone JAR file" and choose the file we just downloaded.

Now close the settings, go back to the Extensions tab, and click on Autorize again. This time you should be able to click the "Install" button:

After installing, there should be a new Autorize tab at the top. Go there and click on "Autorize is off" to toggle it on. 

When you browse to any authenticated or unauthenticated pages, Autorize will now send a separate unauthenticated request to test for a bypass:

For example, some of the graphql requests on HackerOne allow querying without authentication and therefore show up as "Bypassed" on Autorize. Others show as enforced, for example, a request to view the user's own profile will return a different response when unauthenticated.

There are many more awesome Burp extensions that you can install — have a look at this larger list.

Recon & Discovery 

Good recon is half the win; therefore, we outline different areas of tooling setup for each basic type of reconnaissance for web hacking. Note that tooling is constantly evolving, and this is not an exhaustive list by any means; it's just to get you started.

For the purpose of this guide, we are skipping the usual asset discovery tools (such as subdomain enumeration and port scanning) to focus on web applications; for an introduction to asset discovery, you can read this blog from HackerOne.

For service and content discovery tools that run on the command line, you may wish to spin up a small VPS (Virtual Private Server) in the cloud for running those scans since it would return results a lot faster, especially if you are scanning horizontally across a lot of targets.

Technology Stack

When you start testing a web app, you want to get a good feel of what kind of technology stacks it's built on. One way to do this in the browser is via a tool called Wappalyzer, which passively analyzes the website as you browse to detect technologies in use. You can use the extension for Firefox here or the Chrome webstore. 

For example, Wappalyzer just detected that hackerone.com is using Drupal 10 and MariaDB.

We may also wish to detect out-of-date javascript libraries, which may lead to finding bugs like DOM-based cross-site scripting (XSS) and cross-origin resource sharing (CORS) vulnerabilities; a Chrome extension called retire.js can be used for this, which can be installed from the Chrome webstore. 

Service and Vulnerabilities

To further discover what might be running on a web app and what known vulnerabilities it may have, a powerful scanner like nuclei is essential for your hacking environment. You can install nuclei by running:

apt install nuclei

You should run Nuclei at least once initially (just invoke on the shell with nuclei) so that it can install and update nuclei-templates, which are scanning modules written by ProjectDiscover and the wider community. It includes a wide range of scans to detect various technologies, extracting version numbers, finding exposed admin panels, and scanning for known CVEs.

This is how you can run nuclei to scan for known web-based CVEs:

nuclei -target https://example.com -t http/cves/

Or run a workflow specifically targeting Drupal websites using a workflow:

nuclei -target https://example.com -w workflows/drupal-workflow.yaml

Content Discovery

The two popular techniques of content discovery are spidering and directory brute forcing, and we'll get tools to do both of those things.

Spidering follows existing links found on the web app and follows them to detect more links, and so on. A great spidering tool is gospider, which parses site maps, robots.txt, and javascript files along with normal links to find more paths.

It's written in Go, like a lot of fast hacking tools. Let's set up Go in our environment so that we can install Go-based tools quickly in the future.

First, run apt install golang -y on your terminal. Then open your ~/.bashrc file in a text editor and add the following lines:

Now reload .bashrc by running source ~/.bashrc

This will configure your PATH variable so that your shell can find golang tools installed on your system.

You can now install gospider via the go install command:

GO111MODULE=on go install github.com/jaeles-project/gospider@latest

For effective directory brute forcing, you need good wordlists. On Kali, there are a number of preinstalled wordlists in /usr/share/wordlists; however, better word lists, such as ones from wordlists.assetnote.io and the SecLists collection can be used. 

Wordlists from AssetNote are generated based on categories and use cases. Having a self-maintained ~/wordlists directory is a good idea, as you can use it to store all of your own wordlists separate from what Kali has (and you can download AssetNote wordlists based on the target and your needs)

One very fast directory brute-forcing tool you can use is feroxbuster, which is written in Rust. You can follow these installation steps. (hint: apt!) 

This is how to use feroxbuster with the wordlists downloaded from AssetNote:

feroxbuster --url https://example.com --wordlist ~/wordlists/httparchive_apiroutes_2024_01_28.txt

Generating custom wordlists based on each web application helps with discovering hidden paths that default wordlists don't contain. Now that we've set up golang, we can install golang tools easily, such as cook, which is an advanced wordlist generation tool:

go install -v github.com/glitchedgitz/cook/v2/cmd/cook@latest

Cook can be used during testing when interesting keywords and patterns and found to generate wordlists with a combination of words, separators, and file extensions:

Static Analysis Tools

An often overlooked area for web application testing is static analysis of source code. Both client-side and server-side source code can reveal interesting things about the web application that dynamic testing simply does not cover. This includes exposed secrets in source code such as API tokens and passwords, dangerous function calls that lead to XSS, template injection, or unsafe deserialization bugs.

Semgrep

Semgrep is a SAST (Static Application Security Testing) tool that parses source code semantically and uses templates to detect vulnerabilities. For example, using the unsafe parameter when performing serialization in Javascript could lead to XSS, and semgrep can detect that by scanning the source code.

Semgrep can be easily installed as a Python3 pip module:

pip3 install semgrep

You can then use semgrep's publicly available rulesets (ci and owasp-top-ten) to scan a code base, and go through its findings:

semgrep -c p/ci -c p/owasp-top-ten .

Trufflehog

Trufflehog can detect secrets in a variety of places, including source code, S3 buckets, git repositories, and so on. Furthermore, it has built-in verification capabilities that can verify if a secret is valid or not by hitting the service API (for example, getting the username from a leaked GitHub token or ARN from an AWS key). Trufflehog has its own installation script that you can run (or you can manually download it from releases):

To scan a file with trufflehog, simply download the script and run trufflehog.

Decompilers

Sometimes you might get lucky and come across web applications that have binaries available (such as DLL or JAR files) or have a docker image on Docker Hub that contains those files.

In those cases, you may need decompilers. The two most common languages that web applications use that are also usually compiled are Java and C#. To install those languages on your Kali, you can run:

apt install mono-complete default-jdk

To decompile Java, you may want to download these decompilers and keep their JAR files handy for later: 

• jd-gui
• jd-cli
• cfr

You can run JD-GUI like this::

java -jar ~/Downloads/jd-gui-*.jar

To decompile C#, there are two good decompilers to choose from. One is dnSpy (although it is old and unmaintained); another one is dotNetPeek by JetBrains.

Out-of-band Testing Tools

For some vulnerabilities, such as Blind XSS, XML External Entity injection, and Remote File Inclusion, you may need a URL that's internet-exposed to receive the callback payload.

For general DNS or HTTP callbacks, you can use interasch, which will provide a random endpoint you can include in payloads. When it receives an interaction, it will provide the full HTTP and DNS request, including all headers and source IP addresses:

Another tool you can use for XSS callbacks that require signup is bxsshunter. It allows you to create and generate callback links as well as automatically generate payloads for testing:

An open-source, self-hosted alternative is xsshunter, but that requires hosting on a cloud Virtual Private Server (VPS) and a registered domain name, so we're leaving it as an exercise to the reader.

Sometimes you might need to expose a web server with some files for the web application target to download or interact with, such as for testing Remote File Inclusion (RFI). In that case, having a reverse tunnel from localhost to the internet can allow you to quickly set up a callback for any arbitrary web service. You can download cloudflared from its GitHub releases for this purpose.

For example, you may wish to host a PHP file for the target web application to include and run. On one terminal, create the file and start a Python web server on port 8000:

On another terminal, spin up a Cloudflare tunnel:

Now you can use your fresh Cloudflare URL to access the info.php file:

https://replace-with-your-subdomain.trycloudflare.com/info.php

Conclusion

There's a lot covered here, but don't fret — setting up a good environment for web hacking isn't meant to be a quick job; it takes effort, persistence, and time for you, the hacker, to get used to your environment as much as it gets more suited to your skills and style. It's a two-way symbiosis, as you grow into your tools and your tools grow with you; all you have to do is start and not stop.