What are HTTP headers?

HTTP headers are something that we think we know, but it is something that we don’t really spend much time trying to learn. They are really important tools if you really want to reach the full potential of our web apps or web security that you are developing. With that, let’s start by answering the question: what are HTTP headers?

HTTP headers are additional HTTP data being sent and received by both the client program and server on every HTTP request and response. They consist of case-insensitive names followed by a colon (:), then by their corresponding value, and each of them are defined in RFC 4229. These headers are usually invisible to the end-user and are only visible to the backend programs and people maintaining the internet system. 

HTTP headers also define how information sent/received are encoded through the connection (e,g, Accept-Encoding), the session verification and identification of the client (e.g. browser cookies, IP address, user-agent) or their anonymity thereof (e.g. VPN or proxy masking, user-agent spoofing), how the server should handle data (e.g. Do-Not-Track), etc.

Where can we view HTTP headers?

When you open your browser (e.g. Chrome, Firefox, etc.) and type the URL address of any website, your browser will be sending requests to the server and the server will deliver all the content of the website back to the browser, together with all the HTTP headers. You can view and access all the requested content of the website on the main panel of the browser, however you cannot see the HTTP headers immediately.

To view and access the HTTP headers, right click anywhere inside the website page and choose “Inspect” or “Inspect Elements” to open the Developer tools. Inside the Developer tool panel choose the “Network” on the main tab, then select a “Name” on the “Request” panel. You can view all the HTTP headers on the “Headers” tab. The figure below shows where to find the HTTP headers using Chrome.

Let’s use www.google.com as an example. After typing the URL on the address bar of Chrome browser, and then clicking the refresh button, both of these processes are actually doing a GET request to google.com. Then you’ll see a lot of things loaded on the browser, most of these are resources that run the page or scripts. If you click any Name within the request panel, e.g. the topmost www.google.com, you will find some headers there.

For Chrome, you will see that the headers are classified into three categories based on context: the general headers, the response headers, and finally the request headers. On each category you will find different HTTP headers: header names together with their corresponding values.

HTTP Header categorization based on contexts

As mentioned above HTTP headers can be classified based on context. Traditionally, there are 3 main categories based on context, namely the request headers, the response headers, and the general headers. However, recently, the general headers are simply considered as a short summary of selected request and response headers. Recently, HTTP headers are now technically classified into 4 categories based on context, namely, the request headers, the response headers, the representation headers, and the payload headers.

  1. Request headers

    HTTP headers used in an HTTP request that contains information about the request, so that the server can provide a proper response. It can be information about the resource to be fetched, or about the client requesting the resource.

    Here are some examples of request headers:

    GET /home.html HTTP/1.1
    Host: developer.mozilla.org
    User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:50.0) Gecko/20100101 Firefox/50.0
    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
    Accept-Language: en-US,en;q=0.5
    Accept-Encoding: gzip, deflate, br
    Referer: https://developer.mozilla.org/testpage.html
    Connection: keep-alive
    Upgrade-Insecure-Requests: 1
    If-Modified-Since: Mon, 18 Jul 2016 02:36:04 GMT
    If-None-Match: “c561c68d0ba92bbeb8b0fff2a9199f722e3a621a”
    Cache-Control: max-age=0
  1. Response headers

    HTTP headers used in an HTTP response that contains information about the response, which the client has requested. It can be information about its age, its location or the server providing it. 

    Here are some examples of response headers:
    200 OK
    Access-Control-Allow-Origin: *
    Connection: Keep-Alive
    Content-Encoding: gzip
    Content-Type: text/html; charset=utf-8
    Date: Mon, 18 Jul 2016 16:06:00 GMT
    Etag: “c561c68d0ba92bbeb8b0f612a9199f722e3a621a”
    Keep-Alive: timeout=5, max=997
    Last-Modified: Mon, 18 Jul 2016 02:36:04 GMT
    Server: Apache
    Set-Cookie: mykey=myvalue; expires=Mon, 17-Jul-2017 16:06:00 GMT; Max-Age=31449600; Path=/; secure
    Transfer-Encoding: chunked
    Vary: Cookie, Accept-Encoding
    X-Backend-Server: developer2.webapp.scl3.mozilla.com
    X-Cache-Info: not cacheable; meta data too large
    X-kuma-revision: 1085259
    X-frame-options: DENY
  1. Representation headers

    HTTP headers that are present in both HTTP request and response that describes the particular representation of the resource sent in an HTTP message body. It contains information about the body of the resource, like its MIME type, or encoding/compression applied.

    Representations are different versions of a particular resource that might be returned from a request. For example, the same data resource might be formatted as XML or JSON, and that resource might then be encoded in one or more compressed formats for sending. Representation headers may be present in both HTTP request and response messages.

    Here are some examples of representation headers:

    Content-Encoding: gzip
    Content-Type: text/html; charset=utf-8
    Content-Language: de-DE, en-CA
    Content-Location: /my-receipts/38
  1. Payload headers – HTTP headers that describe the payload information related to safe transport and reconstruction of the original resource representation, from one or more messages. It contains representation-independent information about payload data, such as the length of the message payload, which part of the resource is carried in this payload (for a multi-part message), any encoding applied for transport, message integrity checks, etc.

    Payload headers may be present in both HTTP request and response messages.

    Here are some examples of payload headers:

    Content-length: 6553
    Content-Range: bytes 200-1000/67589
    Trailer: Max-Forwards
    Transfer-Encoding: gzip, chunked
  1. General headers – HTTP headers that can be used for both request and response, but don’t apply to the content itself. This type of header classification is already considered outdated, headers are now simply referred to as response or request headers based on context. Some browsers such as Chrome still use this header classification.

    Here are some examples of general headers:
    Request URL: https://www.google.com/log?format=json&hasfast=true&authuser=0
    Request Method: POST
    Status Code: 200 
    Remote Address: 127.0.0.1:8888
    Referrer Policy: origin

HTTP Header categorization based on function

Each HTTP headers serve a specific function on the communication between the client and the server, hence it can be useful for developers to classify HTTP headers based on function.

1. Authentication

WWW-Authenticate
Authorization
Proxy-Authenticate
Proxy-Authorization
2. Caching

Age
Cache-Control
Clear-Site-Data
Expires
Pragma
Warning
3. Conditionals

Last-Modified
ETag
If-Match
If-None-Match
If-Modified-Since
If-Unmodified-Since
Vary
4. Client hints

Accept-CH
Accept-CH-Lifetime  
5. Device client hints

Content-DPR
Device-Memory
DPR
Viewport-Width
Width 
6. Network client hints

Downlink
ECT
RTT
Save-Data 
7. Connection management

Connection
Keep-Alive
8. Content negotiation

Accept
Accept-Encoding
Accept-Language
9. Controls

Expect
Max-Forwards
10. Cookies

Cookie
Set-Cookie
Cookie2
Set-Cookie2 
11. Message body information

Content-Length
Content-Type
Content-Encoding
Content-Language
Content-Location
12. Proxies

Forwarded
X-Forwarded-For
X-Forwarded-Host
X-Forwarded-Proto
Via
13. Downloads

Content-Disposition
14. Range requests

Accept-Ranges
Range
If-Range
Content-Range
15. Redirects

Location
16. Request context

From
Host
Referer
Referrer-Policy
User-Agent
17. Response context

Allow
Server
18. Fetch metadata request headers

Sec-Fetch-Site
Sec-Fetch-Mode
Sec-Fetch-User
Sec-Fetch-Dest
19. HTTP Public Key Pinning (HPKP)

Public-Key-Pins
Public-Key-Pins-Report-Only
20. Server-sent events

Last-Event-ID
NEL Ping-From
Ping-To
Report-To
21. Transfer coding

Transfer-Encoding
TE
Trailer
22. Security

Cross-Origin-Embedder-Policy (COEP)
Cross-Origin-Opener-Policy (COOP)
Cross-Origin-Resource-Policy (CORP)
Content-Security-Policy (CSP)
Content-Security-Policy-Report-Only
Expect-CT
Feature-Policy
Origin-Isolation Strict-Transport-Security (HSTS)
Upgrade-Insecure-Requests
X-Content-Type-Options
X-Download-Options
X-Frame-Options (XFO)
X-Permitted-Cross-Domain-Policies
X-Powered-By
X-XSS-Protection
23. CORS

Access-Control-Allow-Origin
Access-Control-Allow-Credentials
Access-Control-Allow-Headers
Access-Control-Allow-Methods
Access-Control-Expose-Headers
Access-Control-Max-Age
Access-Control-Request-Headers
Access-Control-Request-Method
Origin
Timing-Allow-Origin

24. WebSockets

Sec-WebSocket-Key
Sec-WebSocket-Extensions
Sec-WebSocket-Accept
Sec-WebSocket-Protocol
Sec-WebSocket-Version
25. Other

Accept-Push-Policy
Accept-Signature
Alt-Svc
Date
Early-Data
Large-Allocation
Link
Push-Policy
Retry-After
Signature
Signed-Headers
Server-Timing
Service-Worker-Allowed
SourceMap
Upgrade
X-DNS-Prefetch-Control
X-Firefox-Spdy
X-Pingback
X-Requested-With
X-Robots-Tag
X-UA-Compatible 
Reference: MDN Web Docs