42/42_INT_12_webserv
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

wip parsing request

Browse Source
This commit is contained in:
hugogogo
2022-07-29 16:48:37 +02:00
parent c821d3d92d
commit e132802a8d
2 changed files with 55 additions and 283 deletions
Download Patch File Download Diff File Expand all files Collapse all files

283
README.md
View File

@@ -17,59 +17,6 @@
- **getsockname :** returns the current address to which a socket fd is bound
- **fcntl :** manipulate an open fd, by performing some actions, like duplicate it or changing its flags
---
## correction
[correction](https://github.com/AliMaskar96/42-Correction-Sheets/blob/master/ng_5_webserv.pdf)
#### general
- launch the installation of siege with homebrew
- explain the basics of an HTTP server
- ask which function they used for I/O Multiplexing
- ask to get an explanation of how select (or equivalent) is working
- ask if they use only one select (or equivalent) and how they've managed the server accept and the client read/write
- the select (or equivalent) should be in the main loop and should check fd for read and write AT THE SAME TIME, if not please give a 0 and stop the evaluation
- there should be only one read or one write per client per selct (or equivalent). Ask to show the code that goes from the select (or equivalent) to the read and write of a client
- search for all read/recv/write/send on a socket and check that if an error returned the client is removed
- search for all read/recv/write/send and check if the returned value is well checked. (checking only -1 or 0 is not good, you should check both)
- if a check of errno is done after read/recv/write/send, please stop the evaluation and put a mark to 0
- writing ot reading ANY file descriptor withour going through the select (or equivalent) is strickly FORBIDDEN
#### configuration
- look for the HTTP response status codes list on internet and during this evaluation. if any status codes is wrong don't give related points.
- setup multiple servers with different port
- setup multiple servers with different hostname (use something like: curl --resolve example.com:80:127.0.0.1 http://example.com/)
- setup default error page (try to change the error 404)
- limit the client body (use curl -X POST -H "Content-Type: plain/text" --data "BODY IS HERE write something shorter or longer than body limit")
- setup routes in a server to different directories
- setup a default file to search for if you ask for a directory
- setup a list of method accepted for a certain route (ex: try to delete something with and without permission)
#### basic checks
Using telnet, curl, prepared files demonstrates that the following features work properly:
- GET requests -> should work
- POST requests -> should work
- DELETE requests -> should work
- UNKNOWN requests -> should not produce any crash
- For every test the status code must be good
- upload some file to the server and get it back
#### Check with a browser
- Use the reference browser of the team, open the network part of it and try to connect to the server with it
- Look at the request header and response header
- It should be compatible to serve a fully static website
- Try a wrong URL on the server
- Try to list a directory
- Try a redirected URL
- Try things
#### Port issues
- In the configuration file setup multiple ports and use different websites, use the browser to check that the configuration is working as expected, and show the right website.
- In the configuration try to setup the same port multiple times. It should not work.
- Launch multiple servers at the same time with different configurations but with common ports. Is it working? If it is working, ask why the server should work if one of the configurations isn't working. keep going
#### Siege & stress test
- Use Siege to run some stress tests.
- Availability should be above 99.5% for a simple get on an empty page with a siege -b on that page
- Check if there is no memory leak (monitor the process memory usage it should not go up indefinitely)
- Check if there is no hanging connection
- You should be able to use siege indefinitely without restarting the server (look at siege -b)
---
## todo
@@ -156,237 +103,10 @@ SERVER_SOFTWARE : the server software you're using (e.g. Apache 1.3)
REDIRECT_STATUS : for exemple, 200
```
## http headers
[http headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers)
#### AUTHENTICATION
- WWW-Authenticate : Defines the authentication method that should be used to access a resource.
- Authorization : Contains the credentials to authenticate a user-agent with a server.
- Proxy-Authenticate : Defines the authentication method that should be used to access a resource behind a proxy server.
- Proxy-Authorization : Contains the credentials to authenticate a user agent with a proxy server.
#### CACHING
- Age : The time, in seconds, that the object has been in a proxy cache.
- Cache-Control : Directives for caching mechanisms in both requests and responses.
- Clear-Site-Data : Clears browsing data (e.g. cookies, storage, cache) associated with the requesting website.
- Expires : The date/time after which the response is considered stale.
- Pragma : Implementation-specific header that may have various effects anywhere along the request-response chain. Used for backwards compatibility with HTTP/1.0 caches where the Cache-Control header is not yet present.
- (Warning) : (Deprecated) General warning information about possible problems.
#### CLIENT HINTS
HTTP Client hints are a set of request headers that provide useful information about the client such as device type and network conditions, and allow servers to optimize what is served for those conditions.
Servers proactively requests the client hint headers they are interested in from the client using Accept-CH. The client may then choose to include the requested headers in subsequent requests.
- (Accept-CH) : (Experimental) Servers can advertise support for Client Hints using the Accept-CH header field or an equivalent HTML <meta> element with http-equiv attribute.
- (Accept-CH-Lifetime) : (Experimental Deprecated) Servers can ask the client to remember the set of Client Hints that the server supports for a specified period of time, to enable delivery of Client Hints on subsequent requests to the server's origin.
The different categories of client hints are listed below.
##### User agent client hints
The UA client hints are request headers that provide information about the user agent and the platform/architecture on which it is running:
- (Sec-CH-UA) : (Experimental) User agent's branding and version.
- (Sec-CH-UA-Arch) : (Experimental) User agent's underlying platform architecture.
- (Sec-CH-UA-Bitness) : (Experimental) User agent's underlying CPU architecture bitness (for example "64" bit).
- (Sec-CH-UA-Full-Version) : (Deprecated) User agent's full semantic version string.
- (Sec-CH-UA-Full-Version-List) : (Experimental) Full version for each brand in the user agent's brand list.
- (Sec-CH-UA-Mobile) : (Experimental) User agent is running on a mobile device or, more generally, prefers a "mobile" user experience.
- (Sec-CH-UA-Model) : (Experimental) User agent's device model.
- (Sec-CH-UA-Platform) : (Experimental) User agent's underlying operation system/platform.
- (Sec-CH-UA-Platform-Version) : (Experimental) User agent's underlying operation system version.
##### Device client hints
- (Content-DPR) : (Deprecated Experimental) Response header used to confirm the image device to pixel ratio in requests where the DPR client hint was used to select an image resource.
- (Device-Memory) : (Deprecated Experimental) Approximate amount of available client RAM memory. This is part of the Device Memory API.
- (DPR Deprecated) : (Experimental) Client device pixel ratio (DPR), which is the number of physical device pixels corresponding to every CSS pixel.
- (Viewport-Width) : (Deprecated Experimental) A number that indicates the layout viewport width in CSS pixels. The provided pixel value is a number rounded to the smallest following integer (i.e. ceiling value).
- (Width) : (Deprecated Experimental) A number that indicates the desired resource width in physical pixels (i.e. intrinsic size of an image).
##### Network client hints
Network client hints allow a server to choose what information is sent based on the user choice and network bandwidth and latency.
- Downlink : Approximate bandwidth of the client's connection to the server, in Mbps. This is part of the Network Information API.
- ECT : The effective connection type ("network profile") that best matches the connection's latency and bandwidth. This is part of the Network Information API.
- RTT : Application layer round trip time (RTT) in milliseconds, which includes the server processing time. This is part of the Network Information API.
- (Save-Data) : (Experimental) A boolean that indicates the user agent's preference for reduced data usage.
#### Conditionals
- Last-Modified : The last modification date of the resource, used to compare several versions of the same resource. It is less accurate than ETag, but easier to calculate in some environments. Conditional requests using If-Modified-Since and If-Unmodified-Since use this value to change the behavior of the request.
- ETag : A unique string identifying the version of the resource. Conditional requests using If-Match and If-None-Match use this value to change the behavior of the request.
- If-Match : Makes the request conditional, and applies the method only if the stored resource matches one of the given ETags.
- If-None-Match : Makes the request conditional, and applies the method only if the stored resource doesn't match any of the given ETags. This is used to update caches (for safe requests), or to prevent uploading a new resource when one already exists.
- If-Modified-Since : Makes the request conditional, and expects the resource to be transmitted only if it has been modified after the given date. This is used to transmit data only when the cache is out of date.
- If-Unmodified-Since : Makes the request conditional, and expects the resource to be transmitted only if it has not been modified after the given date. This ensures the coherence of a new fragment of a specific range with previous ones, or to implement an optimistic concurrency control system when modifying existing documents.
- Vary : Determines how to match request headers to decide whether a cached response can be used rather than requesting a fresh one from the origin server.
#### Connection management
- Connection : Controls whether the network connection stays open after the current transaction finishes.
- Keep-Alive : Controls how long a persistent connection should stay open.
#### Content negotiation
Content negotiation headers.
- Accept : Informs the server about the types of data that can be sent back.
- Accept-Encoding : The encoding algorithm, usually a compression algorithm, that can be used on the resource sent back.
- Accept-Language : Informs the server about the human language the server is expected to send back. This is a hint and is not necessarily under the full control of the user: the server should always pay attention not to override an explicit user choice (like selecting a language from a dropdown).
#### Controls
- Expect : Indicates expectations that need to be fulfilled by the server to properly handle the request.
- Max-Forwards : TBD
#### Cookies
- Cookie : Contains stored HTTP cookies previously sent by the server with the Set-Cookie header.
- Set-Cookie : Send cookies from the server to the user-agent.
#### CORS
Learn more about CORS here.
- Access-Control-Allow-Origin : Indicates whether the response can be shared.
- Access-Control-Allow-Credentials : Indicates whether the response to the request can be exposed when the credentials flag is true.
- Access-Control-Allow-Headers : Used in response to a preflight request to indicate which HTTP headers can be used when making the actual request.
- Access-Control-Allow-Methods : Specifies the methods allowed when accessing the resource in response to a preflight request.
- Access-Control-Expose-Headers : Indicates which headers can be exposed as part of the response by listing their names.
- Access-Control-Max-Age : Indicates how long the results of a preflight request can be cached.
- Access-Control-Request-Headers : Used when issuing a preflight request to let the server know which HTTP headers will be used when the actual request is made.
- Access-Control-Request-Method : Used when issuing a preflight request to let the server know which HTTP method will be used when the actual request is made.
- Origin : Indicates where a fetch originates from.
- Timing-Allow-Origin : Specifies origins that are allowed to see values of attributes retrieved via features of the Resource Timing API, which would otherwise be reported as zero due to cross-origin restrictions.
#### Downloads
- Content-Disposition : Indicates if the resource transmitted should be displayed inline (default behavior without the header), or if it should be handled like a download and the browser should present a "Save As" dialog.
#### Message body information
- Content-Length : The size of the resource, in decimal number of bytes.
- Content-Type : Indicates the media type of the resource.
- Content-Encoding : Used to specify the compression algorithm.
- Content-Language : Describes the human language(s) intended for the audience, so that it allows a user to differentiate according to the users' own preferred language.
- Content-Location : Indicates an alternate location for the returned data.
#### Proxies
- Forwarded : Contains information from the client-facing side of proxy servers that is altered or lost when a proxy is involved in the path of the request.
- (X-Forwarded-For) : (Non-Standard) Identifies the originating IP addresses of a client connecting to a web server through an HTTP proxy or a load balancer.
- (X-Forwarded-Host) : (Non-Standard) Identifies the original host requested that a client used to connect to your proxy or load balancer.
- (X-Forwarded-Proto) : (Non-Standard) Identifies the protocol (HTTP or HTTPS) that a client used to connect to your proxy or load balancer.
- Via : Added by proxies, both forward and reverse proxies, and can appear in the request headers and the response headers.
#### Redirects
- Location : Indicates the URL to redirect a page to.
#### Request context
- From : Contains an Internet email address for a human user who controls the requesting user agent.
- Host : Specifies the domain name of the server (for virtual hosting), and (optionally) the TCP port number on which the server is listening.
- Referer : The address of the previous web page from which a link to the currently requested page was followed.
- Referrer-Policy : Governs which referrer information sent in the Referer header should be included with requests made.
- User-Agent : Contains a characteristic string that allows the network protocol peers to identify the application type, operating system, software vendor or software version of the requesting software user agent. See also the Firefox user agent string reference.
#### Response context
- Allow : Lists the set of HTTP request methods supported by a resource.
- Server : Contains information about the software used by the origin server to handle the request.
#### Range requests
- Accept-Ranges : Indicates if the server supports range requests, and if so in which unit the range can be expressed.
- Range : Indicates the part of a document that the server should return.
- If-Range : Creates a conditional range request that is only fulfilled if the given etag or date matches the remote resource. Used to prevent downloading two ranges from incompatible version of the resource.
- Content-Range : Indicates where in a full body message a partial message belongs.
#### Security
- Cross-Origin-Embedder-Policy (COEP) : Allows a server to declare an embedder policy for a given document.
- Cross-Origin-Opener-Policy (COOP) : Prevents other domains from opening/controlling a window.
- Cross-Origin-Resource-Policy (CORP) : Prevents other domains from reading the response of the resources to which this header is applied.
- Content-Security-Policy (CSP) : Controls resources the user agent is allowed to load for a given page.
- Content-Security-Policy-Report-Only : Allows web developers to experiment with policies by monitoring, but not enforcing, their effects. These violation reports consist of JSON documents sent via an HTTP POST request to the specified URI.
- Expect-CT : Allows sites to opt in to reporting and/or enforcement of Certificate Transparency requirements, which prevents the use of misissued certificates for that site from going unnoticed. When a site enables the Expect-CT header, they are requesting that Chrome check that any certificate for that site appears in public CT logs.
- Feature-Policy : Provides a mechanism to allow and deny the use of browser features in its own frame, and in iframes that it embeds.
- (Origin-Isolation) : (Experimental) Provides a mechanism to allow web applications to isolate their origins.
- Strict-Transport-Security (HSTS) : Force communication using HTTPS instead of HTTP.
- Upgrade-Insecure-Requests : Sends a signal to the server expressing the client's preference for an encrypted and authenticated response, and that it can successfully handle the upgrade-insecure-requests directive.
- X-Content-Type-Options : Disables MIME sniffing and forces browser to use the type given in Content-Type.
- X-Download-Options : The X-Download-Options HTTP header indicates that the browser (Internet Explorer) should not display the option to "Open" a file that has been downloaded from an application, to prevent phishing attacks as the file otherwise would gain access to execute in the context of the application.
- X-Frame-Options (XFO) : Indicates whether a browser should be allowed to render a page in a <frame>, <iframe>, <embed> or <object>.
- X-Permitted-Cross-Domain-Policies : Specifies if a cross-domain policy file (crossdomain.xml) is allowed. The file may define a policy to grant clients, such as Adobe's Flash Player (now obsolete), Adobe Acrobat, Microsoft Silverlight (now obsolete), or Apache Flex, permission to handle data across domains that would otherwise be restricted due to the Same-Origin Policy. See the Cross-domain Policy File Specification for more information.
- X-Powered-By : May be set by hosting environments or other frameworks and contains information about them while not providing any usefulness to the application or its visitors. Unset this header to avoid exposing potential vulnerabilities.
- X-XSS-Protection : Enables cross-site scripting filtering.
#### FETCH METADATA REQUEST HEADERS
Fetch metadata request headers provides information about the context from which the request originated. This allows a server to make decisions about whether a request should be allowed based on where the request came from and how the resource will be used.
- Sec-Fetch-Site : It is a request header that indicates the relationship between a request initiator's origin and its target's origin. It is a Structured Header whose value is a token with possible values cross-site, same-origin, same-site, and none.
- Sec-Fetch-Mode : It is a request header that indicates the request's mode to a server. It is a Structured Header whose value is a token with possible values cors, navigate, no-cors, same-origin, and websocket.
- Sec-Fetch-User : It is a request header that indicates whether or not a navigation request was triggered by user activation. It is a Structured Header whose value is a boolean so possible values are ?0 for false and ?1 for true.
- Sec-Fetch-Dest : It is a request header that indicates the request's destination to a server. It is a Structured Header whose value is a token with possible values audio, audioworklet, document, embed, empty, font, image, manifest, object, paintworklet, report, script, serviceworker, sharedworker, style, track, video, worker, and xslt.
- Service-Worker-Navigation-Preload : A request header sent in preemptive request to fetch() a resource during service worker boot. The value, which is set with NavigationPreloadManager.setHeaderValue(), can be used to inform a server that a different resource should be returned than in a normal fetch() operation.
#### Server-sent events
- Last-Event-ID : TBD
- (NEL) : (Experimental) Defines a mechanism that enables developers to declare a network error reporting policy.
- Ping-From : TBD
- Ping-To : TBD
- Report-To : Used to specify a server endpoint for the browser to send warning and error reports to.
#### Transfer coding
- Transfer-Encoding : Specifies the form of encoding used to safely transfer the resource to the user.
- TE : Specifies the transfer encodings the user agent is willing to accept.
- Trailer : Allows the sender to include additional fields at the end of chunked message.
#### WebSockets
- Sec-WebSocket-Key : TBD
- Sec-WebSocket-Extensions : TBD
- Sec-WebSocket-Accept : TBD
- Sec-WebSocket-Protocol : TBD
- Sec-WebSocket-Version : TBD
#### Other
- (Accept-Push-Policy) : (Experimental) A client can express the desired push policy for a request by sending an Accept-Push-Policy header field in the request.
- (Accept-Signature) : (Experimental) A client can send the Accept-Signature header field to indicate intention to take advantage of any available signatures and to indicate what kinds of signatures it supports.
- Alt-Svc : Used to list alternate ways to reach this service.
- **Date** : Contains the date and time at which the message was originated.
- *Early-Data* Experimental : Indicates that the request has been conveyed in TLS early data.
- *Large-Allocation* Deprecated : Tells the browser that the page being loaded is going to want to perform a large allocation.
- **Link** : The Link entity-header field provides a means for serializing one or more links in HTTP headers. It is semantically equivalent to the HTML <link> element.
- *Push-Policy* Experimental : A Push-Policy defines the server behavior regarding push when processing a request.
- **Retry-After** : Indicates how long the user agent should wait before making a follow-up request.
- **Signature** Experimental : The Signature header field conveys a list of signatures for an exchange, each one accompanied by information about how to determine the authority of and refresh that signature.
- **--Signed-Headers** (Experimental) : The Signed-Headers header field identifies an ordered list of response header fields to include in a signature.
- **Server-Timing** : Communicates one or more metrics and descriptions for the given request-response cycle.
- **Service-Worker-Allowed** : Used to remove the path restriction by including this header in the response of the Service Worker script.
- **SourceMap** : Links generated code to a source map.
- **Upgrade** : The relevant RFC document for the Upgrade header field is RFC 7230, section 6.7. The standard establishes rules for upgrading or changing to a different protocol on the current client, server, transport protocol connection. For example, this header standard allows a client to change from HTTP 1.1 to HTTP 2.0, assuming the server decides to acknowledge and implement the Upgrade header field. Neither party is required to accept the terms specified in the Upgrade header field. It can be used in both client and server headers. If the Upgrade header field is specified, then the sender MUST also send the Connection header field with the upgrade option specified. For details on the Connection header field please see section 6.1 of the aforementioned RFC.
- **X-DNS-Prefetch-Control** : Controls DNS prefetching, a feature by which browsers proactively perform domain name resolution on both links that the user may choose to follow as well as URLs for items referenced by the document, including images, CSS, JavaScript, and so forth.
- **-- X-Firefox-Spdy** (Deprecated Non-Standard) : TBD
- **-- X-Pingback** (Non-Standard) : TBD
- **X-Requested-With** : TBD
- **-- X-Robots-Tag** (Non-Standard) : The X-Robots-Tag HTTP header is used to indicate how a web page is to be indexed within public search engine results. The header is effectively equivalent to <meta name="robots" content="…">.
- **-- X-UA-Compatible** (Non-Standard) : Used by Internet Explorer to signal which document mode to use.
---
## ressources
- [correction](https://github.com/AliMaskar96/42-Correction-Sheets/blob/master/ng_5_webserv.pdf)
- [create an http server](https://medium.com/from-the-scratch/http-server-what-do-you-need-to-know-to-build-a-simple-http-server-from-scratch-d1ef8945e4fa)
- [guide to network programming](https://beej.us/guide/bgnet/)
- [same, translated in french](http://vidalc.chez.com/lf/socket.html)
@@ -394,4 +114,5 @@ Fetch metadata request headers provides information about the context from which
- [INADDR_ANY for bind](https://stackoverflow.com/questions/16508685/understanding-inaddr-any-for-socket-programming)
- [hack with CGI](https://www.youtube.com/watch?v=ph6-AKByBU4)
- [http headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers)
- [list of http headers fields](https://en.wikipedia.org/wiki/List_of_HTTP_header_fields)

55
srcs/Webserv.cpp
View File

@@ -241,9 +241,60 @@ int Webserv::_epoll_update(int fd, uint32_t events, int op)
//
void _parse_request(Client *client)
{
std::map<std::string, std::string> req = client->request;
std::string request = client->raw_request;
std::map<std::string, std::string> field = client->request;
// size_t size = request.size();
size_t begin = 0;
size_t end;
size_t len;
req.insert();
// std::string str ("test un: deux\ntest deux: trois quatre\ntest :trois quatre cinq");
// std::string sub;
std::cout << str << "\n\n";
int i = 0;
while (end != std::string::npos)
{
// find first portion, before ':'
end = str.find(':', begin);
len = end - begin;
if (end == std::string::npos)
len = end;
sub = str.substr(begin, len);
std::cout << i << "|" << sub << "\n";
// std::cout << "[begin:" << begin << " - end:" << end << " - len:" << len << "] " << sub << "\n";
begin = end + 1;
// find second portion, until '\n'
end = str.find('\n', begin);
len = end - begin;
if (end == std::string::npos)
len = end;
sub = str.substr(begin, len);
std::cout << i << "|" << sub << "\n";
begin = end + 1;
i++;
}
// for (size_t i = 0; i < size; i++)
// {
// field.insert(request);
// }
// GET /home.html HTTP/1.1
// Host: developer.mozilla.org
// User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:50.0) Gecko/20100101 Firefox/50.0
// Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
// Accept-Language: en-US,en;q=0.5
// Accept-Encoding: gzip, deflate, br
// Referer: https://developer.mozilla.org/testpage.html
// Connection: keep-alive
// Upgrade-Insecure-Requests: 1
// If-Modified-Since: Mon, 18 Jul 2016 02:36:04 GMT
// If-None-Match: "c561c68d0ba92bbeb8b0fff2a9199f722e3a621a"
// Cache-Control: max-age=0
}
void Webserv::_serve_file(Client *client, std::string page)
{