---

man

• htons, htonl, ntohs, ntohl : converts the unsigned short or integer argument between host byte order and network byte order
• poll : waits for one of a set of file descriptors to become ready to perform I/O
  • alternatives : select, epoll (epoll_create, epoll_ctl, epoll_wait), kqueue (kqueue, kevent)
• socket : creates an endpoint for communication and returns a file descriptor that refers to that endpoint
• listen : marks a socket as a passive socket, that is, as a socket that will be used to accept incoming connection requests using accept()
• accept : used with connection-based socket types. It extracts the first connection request on the queue of pending connections for the listening socket, creates a new connected socket, and returns a new file descriptor referring to that socket. The newly created socket is not in the listening state. The original socket is unaffected by this call
• send : (~write) used to transmit a message to another socket. May be used only when the socket is in a connected state (so that the intended recipient is known). The only difference between send() and write() is the presence of flags. With a zero flags argument, send() is equivalent to write()
• recv : (~read) used to receive messages from a socket. May be used to receive data on both connectionless and connection-oriented sockets. The only difference between recv() and read() is the presence of flags. With a zero flags argument, recv() is generally equivalent to read()
• bind : associate a socket fd to a local address. When a socket is created with socket(), it exists in a name space (address family) but has no address assigned to it. It is normally necessary to assign a local address using bind() before a socket may receive connections (see accept())
• connect : connects a socket fd to a remote address
• inet_addr : converts the Internet host address cp from IPv4 numbers-and-dots notation into binary data in network byte order. Use of this function is problematic because in case of error it returns -1, wich is a valid address (255.255.255.255). Avoid its use in favor of inet_aton(), inet_pton(), or getaddrinfo()
• setsockopt : manipulate options for a socket fd. Options may exist at multiple protocol levels; they are always present at the uppermost socket level
• getsockname : returns the current address to which a socket fd is bound
• fcntl : manipulate an open fd, by performing some actions, like duplicate it or changing its flags

---

correction

correction

general

• launch the installation of siege with homebrew
• explain the basics of an HTTP server
• ask which function they used for I/O Multiplexing
• ask to get an explanation of how select (or equivalent) is working
• ask if they use only one select (or equivalent) and how they've managed the server accept and the client read/write
• the select (or equivalent) should be in the main loop and should check fd for read and write AT THE SAME TIME, if not please give a 0 and stop the evaluation
• there should be only one read or one write per client per selct (or equivalent). Ask to show the code that goes from the select (or equivalent) to the read and write of a client
• search for all read/recv/write/send on a socket and check that if an error returned the client is removed
• search for all read/recv/write/send and check if the returned value is well checked. (checking only -1 or 0 is not good, you should check both)
• if a check of errno is done after read/recv/write/send, please stop the evaluation and put a mark to 0
• writing ot reading ANY file descriptor withour going through the select (or equivalent) is strickly FORBIDDEN

configuration

• look for the HTTP response status codes list on internet and during this evaluation. if any status codes is wrong don't give related points.
• setup multiple servers with different port
• setup multiple servers with different hostname (use something like: curl --resolve example.com:80:127.0.0.1 http://example.com/)
• setup default error page (try to change the error 404)
• limit the client body (use curl -X POST -H "Content-Type: plain/text" --data "BODY IS HERE write something shorter or longer than body limit")
• setup routes in a server to different directories
• setup a default file to search for if you ask for a directory
• setup a list of method accepted for a certain route (ex: try to delete something with and without permission)

basic checks

Using telnet, curl, prepared files demonstrates that the following features work properly:

• GET requests -> should work
• POST requests -> should work
• DELETE requests -> should work
• UNKNOWN requests -> should not produce any crash
• For every test the status code must be good
• upload some file to the server and get it back

Check with a browser

• Use the reference browser of the team, open the network part of it and try to connect to the server with it
• Look at the request header and response header
• It should be compatible to serve a fully static website
• Try a wrong URL on the server
• Try to list a directory
• Try a redirected URL
• Try things

Port issues

• In the configuration file setup multiple ports and use different websites, use the browser to check that the configuration is working as expected, and show the right website.
• In the configuration try to setup the same port multiple times. It should not work.
• Launch multiple servers at the same time with different configurations but with common ports. Is it working? If it is working, ask why the server should work if one of the configurations isn't working. keep going

Siege & stress test

• Use Siege to run some stress tests.
• Availability should be above 99.5% for a simple get on an empty page with a siege -b on that page
• Check if there is no memory leak (monitor the process memory usage it should not go up indefinitely)
• Check if there is no hanging connection
• You should be able to use siege indefinitely without restarting the server (look at siege -b)

---

todo

• read the RFC and do some tests with telnet and NGINX

parsing config

• Your program has to take a configuration file as argument, or use a default path.
• Choose the port and host of each ’server’.
• Setup the server_names or not.
• The first server for a host:port will be the default for this host:port (that means it will answer to all the requests that don’t belong to an other server).
• Setup default error pages.
• Limit client body size.
• Setup routes with one or multiple of the following rules/configuration (routes wont be using regexp):
  • Define a list of accepted HTTP methods for the route.
  • Define a HTTP redirection.
  • Define a directory or a file from where the file should be searched (for example, if url /kapouet is rooted to /tmp/www, url /kapouet/pouic/toto/pouet is /tmp/www/pouic/toto/pouet).
  • Turn on or off directory listing.
  • Set a default file to answer if the request is a directory.
  • Execute CGI based on certain file extension (for example .php).
  • Make the route able to accept uploaded files and configure where they should be saved.

connection basic

• You can’t execve another web server.
• Your server must never block and the client can be bounced properly if necessary.
• It must be non-blocking and use only 1 poll() (or equivalent) for all the I/O operations between the client and the server (listen included).
• poll() (or equivalent) must check read and write at the same time.
• You must never do a read or a write operation without going through poll() (or equivalent).
• Checking the value of errno is strictly forbidden after a read or a write operation.
• You don’t need to use poll() (or equivalent) before reading your configuration file. Because you have to use non-blocking file descriptors, it is possible to use read/recv or write/send functions with no poll() (or equivalent), and your server wouldn’t be blocking. But it would consume more system resources. Thus, if you try to read/recv or write/send in any file descriptor without using poll() (or equivalent), your grade will be 0.
• You can use every macro and define like FD_SET, FD_CLR, FD_ISSET, FD_ZERO (understanding what and how they do it is very useful).
• A request to your server should never hang forever.
• Your server must be compatible with the web browser of your choice.

parsing request HTTP (fields, ...)

• We will consider that NGINX is HTTP 1.1 compliant and may be used to compare headers and answer behaviors.

response HTTP (fields, ...)

• Your HTTP response status codes must be accurate.
• You server must have default error pages if none are provided.
• You can’t use fork for something else than CGI (like PHP, or Python, and so forth).
• You must be able to serve a fully static website.

upload files

• Clients must be able to upload files.

CGI

• You need at least GET, POST, and DELETE methods.
• Do you wonder what a CGI is?
• Because you won’t call the CGI directly, use the full path as PATH_INFO.
• Just remember that, for chunked request, your server needs to unchunked it and the CGI will expect EOF as end of the body.
• Same things for the output of the CGI. If no content_length is returned from the CGI, EOF will mark the end of the returned data.
• Your program should call the CGI with the file requested as first argument.
• The CGI should be run in the correct directory for relative path file access.
• Your server should work with one CGI (php-CGI, Python, and so forth).

write tests

• Stress tests your server. It must stay available at all cost.
• Do not test with only one program.
• Write your tests with a more convenient language such as Python or Golang, and so forth. Even in C or C++ if you want to

persistent connexion

• Your server must be able to listen to multiple ports (see Configuration file)
• Your server should never die.

---

cgi env variables

cgi env variables wikipedia variables environnements cgi cgi server variables on adobe

AUTH_TYPE			: if the srcipt is protected, the authentification method used to validate the user
CONTENT_LENGTH		: length of the request content
CONTENT_TYPE		: if there is attached information, as with method POST or PUT, this is the content type of the data (e.g. "text/plain", it is set by the attribute "enctype" in html <form> as three values : "application/x-www-form-urlencoded", "multipart/form-data", "text/plain")
GATEWAY_INTERFACE	: CGI version (e.g. CGI/1.1)
PATH_INFO			: if any, path of the resquest in addition to the cgi script path (e.g. for cgi script path = "/usr/web/cgi-bin/script.cgi", and the url = "http://server.org/cgi-bin/script.cgi/house", the PATH-INFO would be "house")
PATH_TRANSLATED		: full path of the request, like path-to-cgi/PATH_INFO, null if PATH_INFO is null (e.g. for "http://server.org/cgi-bin/prog/the/path", PATH_INFO would be : "/the/path" and PATH_TRANSLATED would be : "/usr/web/cgi-bin/prog/the/path")
QUERY_STRING		: everything following the ? in the url sent by client (e.g. for url "http://server.org/query?var1=val2&var2=val2", it would be : "var1=val2&var2=val2")
REMOTE_ADDR			: ip address of the client
REMOTE_HOST			: host name of the client, empty if not known, or equal to REMOTE_ADDR
REMOTE_IDENT		: if known, username of the client, otherwise empty, use for logging only
REMOTE_USER			: username of client, if script is protected and the server support user authentification
REQUEST_METHOD		: method used for the request (for http, usually POST or GET)
SCRIPT_NAME			: path to the cgi, relative to the root, used for self-referencing URLs (e.g. "/cgi-bin/script.cgi")
SERVER_NAME			: name of the server, as hostname, IP address, or DNS (e.g. dns : "www.server.org")
SERVER_PORT			: the port number your server is listening on (e.g. 80)
SERVER_PROTOCOL		: protocol used for the request (e.g. HTTP/1.1)
SERVER_SOFTWARE		: the server software you're using (e.g. Apache 1.3)

redirect status for php-cgi

REDIRECT_STATUS		: for exemple, 200

http headers

http headers

AUTHENTICATION

• WWW-Authenticate : Defines the authentication method that should be used to access a resource.
• Authorization : Contains the credentials to authenticate a user-agent with a server.
• Proxy-Authenticate : Defines the authentication method that should be used to access a resource behind a proxy server.
• Proxy-Authorization : Contains the credentials to authenticate a user agent with a proxy server.

CACHING

• Age : The time, in seconds, that the object has been in a proxy cache.
• Cache-Control : Directives for caching mechanisms in both requests and responses.
• Clear-Site-Data : Clears browsing data (e.g. cookies, storage, cache) associated with the requesting website.
• Expires : The date/time after which the response is considered stale.
• Pragma : Implementation-specific header that may have various effects anywhere along the request-response chain. Used for backwards compatibility with HTTP/1.0 caches where the Cache-Control header is not yet present.
• (Warning) : (Deprecated) General warning information about possible problems.

CLIENT HINTS

HTTP Client hints are a set of request headers that provide useful information about the client such as device type and network conditions, and allow servers to optimize what is served for those conditions. Servers proactively requests the client hint headers they are interested in from the client using Accept-CH. The client may then choose to include the requested headers in subsequent requests.

• (Accept-CH) : (Experimental) Servers can advertise support for Client Hints using the Accept-CH header field or an equivalent HTML element with http-equiv attribute.
• (Accept-CH-Lifetime) : (Experimental Deprecated) Servers can ask the client to remember the set of Client Hints that the server supports for a specified period of time, to enable delivery of Client Hints on subsequent requests to the server's origin.

The different categories of client hints are listed below.

User agent client hints

The UA client hints are request headers that provide information about the user agent and the platform/architecture on which it is running:

• (Sec-CH-UA) : (Experimental) User agent's branding and version.
• (Sec-CH-UA-Arch) : (Experimental) User agent's underlying platform architecture.
• (Sec-CH-UA-Bitness) : (Experimental) User agent's underlying CPU architecture bitness (for example "64" bit).
• (Sec-CH-UA-Full-Version) : (Deprecated) User agent's full semantic version string.
• (Sec-CH-UA-Full-Version-List) : (Experimental) Full version for each brand in the user agent's brand list.
• (Sec-CH-UA-Mobile) : (Experimental) User agent is running on a mobile device or, more generally, prefers a "mobile" user experience.
• (Sec-CH-UA-Model) : (Experimental) User agent's device model.
• (Sec-CH-UA-Platform) : (Experimental) User agent's underlying operation system/platform.
• (Sec-CH-UA-Platform-Version) : (Experimental) User agent's underlying operation system version.

Device client hints

• (Content-DPR) : (Deprecated Experimental) Response header used to confirm the image device to pixel ratio in requests where the DPR client hint was used to select an image resource.
• (Device-Memory) : (Deprecated Experimental) Approximate amount of available client RAM memory. This is part of the Device Memory API.
• (DPR Deprecated) : (Experimental) Client device pixel ratio (DPR), which is the number of physical device pixels corresponding to every CSS pixel.
• (Viewport-Width) : (Deprecated Experimental) A number that indicates the layout viewport width in CSS pixels. The provided pixel value is a number rounded to the smallest following integer (i.e. ceiling value).
• (Width) : (Deprecated Experimental) A number that indicates the desired resource width in physical pixels (i.e. intrinsic size of an image).

Network client hints

Network client hints allow a server to choose what information is sent based on the user choice and network bandwidth and latency.

• Downlink : Approximate bandwidth of the client's connection to the server, in Mbps. This is part of the Network Information API.
• ECT : The effective connection type ("network profile") that best matches the connection's latency and bandwidth. This is part of the Network Information API.
• RTT : Application layer round trip time (RTT) in milliseconds, which includes the server processing time. This is part of the Network Information API.
• (Save-Data) : (Experimental) A boolean that indicates the user agent's preference for reduced data usage.

Conditionals

• Last-Modified : The last modification date of the resource, used to compare several versions of the same resource. It is less accurate than ETag, but easier to calculate in some environments. Conditional requests using If-Modified-Since and If-Unmodified-Since use this value to change the behavior of the request.
• ETag : A unique string identifying the version of the resource. Conditional requests using If-Match and If-None-Match use this value to change the behavior of the request.
• If-Match : Makes the request conditional, and applies the method only if the stored resource matches one of the given ETags.
• If-None-Match : Makes the request conditional, and applies the method only if the stored resource doesn't match any of the given ETags. This is used to update caches (for safe requests), or to prevent uploading a new resource when one already exists.
• If-Modified-Since : Makes the request conditional, and expects the resource to be transmitted only if it has been modified after the given date. This is used to transmit data only when the cache is out of date.
• If-Unmodified-Since : Makes the request conditional, and expects the resource to be transmitted only if it has not been modified after the given date. This ensures the coherence of a new fragment of a specific range with previous ones, or to implement an optimistic concurrency control system when modifying existing documents.
• Vary : Determines how to match request headers to decide whether a cached response can be used rather than requesting a fresh one from the origin server.

Connection management

• Connection : Controls whether the network connection stays open after the current transaction finishes.
• Keep-Alive : Controls how long a persistent connection should stay open.

Content negotiation

Content negotiation headers.

• Accept : Informs the server about the types of data that can be sent back.
• Accept-Encoding : The encoding algorithm, usually a compression algorithm, that can be used on the resource sent back.
• Accept-Language : Informs the server about the human language the server is expected to send back. This is a hint and is not necessarily under the full control of the user: the server should always pay attention not to override an explicit user choice (like selecting a language from a dropdown).

Controls

• Expect : Indicates expectations that need to be fulfilled by the server to properly handle the request.
• Max-Forwards : TBD

Cookies

• Cookie : Contains stored HTTP cookies previously sent by the server with the Set-Cookie header.
• Set-Cookie : Send cookies from the server to the user-agent.

CORS

Learn more about CORS here.

• Access-Control-Allow-Origin : Indicates whether the response can be shared.
• Access-Control-Allow-Credentials : Indicates whether the response to the request can be exposed when the credentials flag is true.
• Access-Control-Allow-Headers : Used in response to a preflight request to indicate which HTTP headers can be used when making the actual request.
• Access-Control-Allow-Methods : Specifies the methods allowed when accessing the resource in response to a preflight request.
• Access-Control-Expose-Headers : Indicates which headers can be exposed as part of the response by listing their names.
• Access-Control-Max-Age : Indicates how long the results of a preflight request can be cached.
• Access-Control-Request-Headers : Used when issuing a preflight request to let the server know which HTTP headers will be used when the actual request is made.
• Access-Control-Request-Method : Used when issuing a preflight request to let the server know which HTTP method will be used when the actual request is made.
• Origin : Indicates where a fetch originates from.
• Timing-Allow-Origin : Specifies origins that are allowed to see values of attributes retrieved via features of the Resource Timing API, which would otherwise be reported as zero due to cross-origin restrictions.

Downloads

• Content-Disposition : Indicates if the resource transmitted should be displayed inline (default behavior without the header), or if it should be handled like a download and the browser should present a "Save As" dialog.

Message body information

• Content-Length : The size of the resource, in decimal number of bytes.
• Content-Type : Indicates the media type of the resource.
• Content-Encoding : Used to specify the compression algorithm.
• Content-Language : Describes the human language(s) intended for the audience, so that it allows a user to differentiate according to the users' own preferred language.
• Content-Location : Indicates an alternate location for the returned data.

Proxies

• Forwarded : Contains information from the client-facing side of proxy servers that is altered or lost when a proxy is involved in the path of the request.
• (X-Forwarded-For) : (Non-Standard) Identifies the originating IP addresses of a client connecting to a web server through an HTTP proxy or a load balancer.
• (X-Forwarded-Host) : (Non-Standard) Identifies the original host requested that a client used to connect to your proxy or load balancer.
• (X-Forwarded-Proto) : (Non-Standard) Identifies the protocol (HTTP or HTTPS) that a client used to connect to your proxy or load balancer.
• Via : Added by proxies, both forward and reverse proxies, and can appear in the request headers and the response headers.

Redirects

• Location : Indicates the URL to redirect a page to.

Request context

• From : Contains an Internet email address for a human user who controls the requesting user agent.
• Host : Specifies the domain name of the server (for virtual hosting), and (optionally) the TCP port number on which the server is listening.
• Referer : The address of the previous web page from which a link to the currently requested page was followed.
• Referrer-Policy : Governs which referrer information sent in the Referer header should be included with requests made.
• User-Agent : Contains a characteristic string that allows the network protocol peers to identify the application type, operating system, software vendor or software version of the requesting software user agent. See also the Firefox user agent string reference.

Response context

• Allow : Lists the set of HTTP request methods supported by a resource.
• Server : Contains information about the software used by the origin server to handle the request.

Range requests

• Accept-Ranges : Indicates if the server supports range requests, and if so in which unit the range can be expressed.
• Range : Indicates the part of a document that the server should return.
• If-Range : Creates a conditional range request that is only fulfilled if the given etag or date matches the remote resource. Used to prevent downloading two ranges from incompatible version of the resource.
• Content-Range : Indicates where in a full body message a partial message belongs.

Security

• Cross-Origin-Embedder-Policy (COEP) : Allows a server to declare an embedder policy for a given document.
• Cross-Origin-Opener-Policy (COOP) : Prevents other domains from opening/controlling a window.
• Cross-Origin-Resource-Policy (CORP) : Prevents other domains from reading the response of the resources to which this header is applied.
• Content-Security-Policy (CSP) : Controls resources the user agent is allowed to load for a given page.
• Content-Security-Policy-Report-Only : Allows web developers to experiment with policies by monitoring, but not enforcing, their effects. These violation reports consist of JSON documents sent via an HTTP POST request to the specified URI.
• Expect-CT : Allows sites to opt in to reporting and/or enforcement of Certificate Transparency requirements, which prevents the use of misissued certificates for that site from going unnoticed. When a site enables the Expect-CT header, they are requesting that Chrome check that any certificate for that site appears in public CT logs.
• Feature-Policy : Provides a mechanism to allow and deny the use of browser features in its own frame, and in iframes that it embeds.
• (Origin-Isolation) : (Experimental) Provides a mechanism to allow web applications to isolate their origins.
• Strict-Transport-Security (HSTS) : Force communication using HTTPS instead of HTTP.
• Upgrade-Insecure-Requests : Sends a signal to the server expressing the client's preference for an encrypted and authenticated response, and that it can successfully handle the upgrade-insecure-requests directive.
• X-Content-Type-Options : Disables MIME sniffing and forces browser to use the type given in Content-Type.
• X-Download-Options : The X-Download-Options HTTP header indicates that the browser (Internet Explorer) should not display the option to "Open" a file that has been downloaded from an application, to prevent phishing attacks as the file otherwise would gain access to execute in the context of the application.
• X-Frame-Options (XFO) : Indicates whether a browser should be allowed to render a page in a ,