Skip to content

A multithreaded, IO multiplexing proxy web server that caches objects using LRU replacement

License

Notifications You must be signed in to change notification settings

bbright024/proxy-web-server

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

74 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

##########
The Final Edit was a lie!

These later updates have been about trying to write better test code.  I wrote & finished this proxy before understanding unit testing, and decided to go back and fill stuff in for practice.
While I was writing the proxy originally, I deleted all the printf() statements and test stuff after everything worked, because I was dumb.
With the exception of LinkedList.c and HashTable.c, the testing of proxy was left up to the driver provided by CMU, and I couldn't figure out how to integrate that script with lcov - something
weird was going on with stdout and stdin redirection and it caused the driver.sh to deadlock whenever run with code compiled for lcov/gcov.  This later stuff is my unit testing practice, cause it's fun
trying to get code coverage to 100%.

Current coverage:
~76%

#########
FINAL EDIT - Tuesday Mar 13 2018
#########
I've decided to restart this project, in order to fix some annoyancies and improve security. Having a single process for all threads is super fast, but it won't scale at all and a compromise in one thread compromises them all.  My proxy 2.0 will be apache / OKWS style, with a master process using RPC to dispatch requests to a pool of worker processes, dynamically scaling them to a max limit.
I will also use this reboot time to fix up my HashTable and LinkedList library to better support fine-grain locking (maybe even lock free someday?) and to fully implement a few other libraries (com_err, bstring, zlog) that will help keep order.  Also, it'll let me do comments and unit testing a lot better than I presently have.
If any job recruiters are reading this, well, sorry you have to see version 1.0 of my proxy.  It'll be back soon in 2.0!

What I learned:
  - How important unit testing/small testing is for seeing dependencies in your code tree
           - and how long it takes to write proper big testing
  - A lot of tricks for Makefiles, like the dep rule, tags rule, and patsubs to get all source code
  - how to properly organize a project directory
  - integration of libraries is trickyyyyy
  - valgrind --tool=helgrind is great for multithreaded programs - finds races and doesn't throw a fit when a thread detatches
  - parsing and sanitizing input strings is HARD
  
-Brian Bright





####
Log:
####

Mon Mar 12
  - gonna work on fine grain locking and reader/writer locks for cache
  - at the same time, get rid of passing structs by value in the hash/ll library
  

####################
TODO:
###################

random fixes:
       - have better lock cleanup after syscall errors if you dont want to close entire server if one thread bugs
       - fix dangling \r\n when saving 'type' in http
       - finish making fully HTTP 1.0 compliant
       - check signed/unsigned conversion, especially overflows
       - add fine grain locking - per bucket locks, also implement reader writer lock paradigm in hash table lib
       - fix char * that should be const char *
 *  - modify code a bit to be able to run AFL
   

Long term:
     - add zlog logging library
     - add bstring library
     - add com_err error handling library
     - split the process up, use RPC to communicate (like OK web server)
     - add a menu to
       * change port of the proxy
       * print out list of ppl who have accessed the proxy
       * list of objects in the cache
       * option to list it all in a browser in html
####################################################################
Brian's Proxy Web Server

This project contains a caching, posix-multithreaded HTTP 1.0 proxy web server.
The server uses IO multiplexing to respond to keyboard input with info on
the cache.

The cache uses a doubly linked list for LRU replacement when it fills up,
and uses a chaining hash table in which the keys are a hash of the host/filename,
and the values are pointers to a struct containing metadata about the file.

To try it out, use 'make' and './bin/proxy 8080' and change settings in Mozilla
Firefox to use a manual proxy on port 8080.

Current limitations:
	-no HTTPS
	-no PUT or HEAD, only GET

Current command line commands (for while the proxy is running in the foreground): 
	p - print out metadata of current objects in cache

	<all other chars> - tells proxy to exit gracefully

The cache uses a LinkedList and HashTable API that I coded from an assignment
in UW's CSE 333.


###########
Other Files
###########

./tiny
    Tiny Web server from the CS:APP text - used by the driver grader
    
./tests/free-port.sh
    Handy script that identifies an unused TCP port that you can use
    for your proxy or tiny. 
    usage: ./free-port.sh

./driver.sh
    The autograder for Basic, Concurrency, and Cache.
    usage: ./driver.sh

./tests/nop-server.py
     helper for the autograder.         

############
The Makefile
############

make          - create an optimized version

make coverage - create a coverage version that automatically executes
     	        following compilation, and after some manual user action
		automatically runs lcov and genhtml to put coverage info
		in readable form in ./build/cov_html
		
make tests    - runs tests automatically

make clean    - deletes compilied libraries/binaries and coverage info.




About

A multithreaded, IO multiplexing proxy web server that caches objects using LRU replacement

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published