HTTP headers are something that we think we know, but it is something that we don’t really spend much time trying to learn. They are really important tools if you really want to reach the full potential of our web apps or web security that you are developing. With that, let’s start by answering the question: what are HTTP headers?
HTTP headers are additional HTTP data being sent and received by both the client program and server on every HTTP request and response. They consist of case-insensitive names followed by a colon (:), then by their corresponding value, and each of them are defined in RFC 4229. These headers are usually invisible to the end-user and are only visible to the backend programs and people maintaining the internet system.
HTTP headers also define how information sent/received are encoded through the connection (e,g, Accept-Encoding), the session verification and identification of the client (e.g. browser cookies, IP address, user-agent) or their anonymity thereof (e.g. VPN or proxy masking, user-agent spoofing), how the server should handle data (e.g. Do-Not-Track), etc.
Where can we view HTTP headers?
When you open your browser (e.g. Chrome, Firefox, etc.) and type the URL address of any website, your browser will be sending requests to the server and the server will deliver all the content of the website back to the browser, together with all the HTTP headers. You can view and access all the requested content of the website on the main panel of the browser, however you cannot see the HTTP headers immediately.
To view and access the HTTP headers, right click anywhere inside the website page and choose “Inspect” or “Inspect Elements” to open the Developer tools. Inside the Developer tool panel choose the “Network” on the main tab, then select a “Name” on the “Request” panel. You can view all the HTTP headers on the “Headers” tab. The figure below shows where to find the HTTP headers using Chrome.
Let’s use www.google.com as an example. After typing the URL on the address bar of Chrome browser, and then clicking the refresh button, both of these processes are actually doing a GET request to google.com. Then you’ll see a lot of things loaded on the browser, most of these are resources that run the page or scripts. If you click any Name within the request panel, e.g. the topmost www.google.com, you will find some headers there.
For Chrome, you will see that the headers are classified into three categories based on context: the general headers, the response headers, and finally the request headers. On each category you will find different HTTP headers: header names together with their corresponding values.
HTTP Header categorization based on contexts
As mentioned above HTTP headers can be classified based on context. Traditionally, there are 3 main categories based on context, namely the request headers, the response headers, and the general headers. However, recently, the general headers are simply considered as a short summary of selected request and response headers. Recently, HTTP headers are now technically classified into 4 categories based on context, namely, the request headers, the response headers, the representation headers, and the payload headers.
- Request headers
HTTP headers used in an HTTP request that contains information about the request, so that the server can provide a proper response. It can be information about the resource to be fetched, or about the client requesting the resource.
Here are some examples of request headers:
GET /home.html HTTP/1.1
Host: developer.mozilla.org
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:50.0) Gecko/20100101 Firefox/50.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://developer.mozilla.org/testpage.html
Connection: keep-alive
Upgrade-Insecure-Requests: 1
If-Modified-Since: Mon, 18 Jul 2016 02:36:04 GMT
If-None-Match: “c561c68d0ba92bbeb8b0fff2a9199f722e3a621a”
Cache-Control: max-age=0
- Response headers
HTTP headers used in an HTTP response that contains information about the response, which the client has requested. It can be information about its age, its location or the server providing it.
Here are some examples of response headers:
200 OK
Access-Control-Allow-Origin: *
Connection: Keep-Alive
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Date: Mon, 18 Jul 2016 16:06:00 GMT
Etag: “c561c68d0ba92bbeb8b0f612a9199f722e3a621a”
Keep-Alive: timeout=5, max=997
Last-Modified: Mon, 18 Jul 2016 02:36:04 GMT
Server: Apache
Set-Cookie: mykey=myvalue; expires=Mon, 17-Jul-2017 16:06:00 GMT; Max-Age=31449600; Path=/; secure
Transfer-Encoding: chunked
Vary: Cookie, Accept-Encoding
X-Backend-Server: developer2.webapp.scl3.mozilla.com
X-Cache-Info: not cacheable; meta data too large
X-kuma-revision: 1085259
X-frame-options: DENY
- Representation headers
HTTP headers that are present in both HTTP request and response that describes the particular representation of the resource sent in an HTTP message body. It contains information about the body of the resource, like its MIME type, or encoding/compression applied.
Representations are different versions of a particular resource that might be returned from a request. For example, the same data resource might be formatted as XML or JSON, and that resource might then be encoded in one or more compressed formats for sending. Representation headers may be present in both HTTP request and response messages.
Here are some examples of representation headers:
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Content-Language: de-DE, en-CA
Content-Location: /my-receipts/38
- Payload headers – HTTP headers that describe the payload information related to safe transport and reconstruction of the original resource representation, from one or more messages. It contains representation-independent information about payload data, such as the length of the message payload, which part of the resource is carried in this payload (for a multi-part message), any encoding applied for transport, message integrity checks, etc.
Payload headers may be present in both HTTP request and response messages.
Here are some examples of payload headers:
Content-length: 6553
Content-Range: bytes 200-1000/67589
Trailer: Max-Forwards
Transfer-Encoding: gzip, chunked
- General headers – HTTP headers that can be used for both request and response, but don’t apply to the content itself. This type of header classification is already considered outdated, headers are now simply referred to as response or request headers based on context. Some browsers such as Chrome still use this header classification.
Here are some examples of general headers:
Request URL: https://www.google.com/log?format=json&hasfast=true&authuser=0
Request Method: POST
Status Code: 200
Remote Address: 127.0.0.1:8888
Referrer Policy: origin
HTTP Header categorization based on function
Each HTTP headers serve a specific function on the communication between the client and the server, hence it can be useful for developers to classify HTTP headers based on function.