Web Reverse Engineering Challenge #

GitHub repository for the challenge I created.

Introduction #

This fall break, I returned to my old high school to teach my high school’s AP Computer Science (APCS) Class. The teacher is an old mentor of mine from high school. He gave me free reign over what I could teach about, so I wanted to go above and beyond.

I have done a lot of web scraping in the past with requests/bs4 and Selenium, but only in the past year did I get into fully simulating the interactions between the web client and server for applications that require authentication. Even for a high schooler, Selenium is trivial to learn, but really understanding what’s going on when a client authenticates with a server requires a little finesse. My goal for this class was to teach the basic intuition for replicating api requests with automated tools such as cURL or requests.

Lesson Plan #

I wrote a flask web server with two different challenges for the students to work through.

The first challenge is a demonstration of how to query an unprotected api. The goal here is to become familiar with basic http requests and the kinds of information that get sent between the client and the server.

The steps to solve are:

1. Go to the first challenge web page, where a table is updated with data from an api request
2. Open the Chrome developer tools and observe the network request
3. Copy and paste the query URL into cURL to retrieve the api response
4. Observe the formatting of the response in JSON
5. Observe the syntax of the URL and experiment with changing the integer input to the only URL parameter
6. Understand that URL parameters are used to, well, parameterize the data that we request from the server

The second challenge is a more realistic representation of an automated web scraping task. The goal here is to understand how to simulate user authentication.

The steps to solve are:

1. Go to the second challenge web page, which is a login screen
2. Open the network tab of the Chrome developer tools and login to the dashboard
3. Observe the URL parameters of the login request.
   • See that the username and password are given to the server.
   • See that a cookie string is returned from the server in the response.
4. Observe the second URL request. See that the cookie is included in the request headers.
5. Simulate the same login request with cURL by passing the username and password as parameters to the correct api endpoint. Retrieve the cookie from the response.
6. Simulate the authenticated api request with cURL by setting the cookie string to the cookie with the -c flag.

How it went #

I was impressed by how quickly the class caught on to the ideas I presented to them. The students appeared to enjoy learning the material, and went out of their way to ask critical and well-thought through questions about the impact of my statements. They ecspecially were curious about how what I was teaching related to hacking techniques.

Something I’m personally proud of from this was the live display I created for the web server. I wanted the students to really get what was happening on the web server in addition to on their web clients. I wrote about a dozen different unique messages for different kinds of events on the web server that print out the actual URL of the request, then explains the specific reason why the request failed or succeeded. It was pretty neat to see the spam of messages on the lab projector screen as the students played with cURL and tried different things. At the end of the class, the server had counted up to hundreds of requests that the students had individually typed out.