Encoding: The Invisible Layer Under Every Web Request

Encoding is one of those topics developers encounter constantly but rarely study directly. Why does a space become %20 in a URL? Why do email attachments contain a wall of random letters? Why does alert('hello') sometimes appear as alert('hello') in HTML? All of these are encoding in action — different systems transforming data into safe, transmissible representations.

Understanding encoding prevents bugs, security vulnerabilities, and hours of confusing debugging. Let's break down the three types you'll encounter most.

1. URL Encoding (Percent Encoding)

URLs can only contain a limited set of ASCII characters. Characters outside this set — spaces, ampersands, Chinese characters, emoji — must be encoded as a percent sign followed by two hexadecimal digits representing the character's byte value.

Original:  https://example.com/search?q=hello world&lang=en
Encoded:   https://example.com/search?q=hello%20world&lang=en

Original:  café au lait
Encoded:   caf%C3%A9%20au%20lait

Note that é becomes %C3%A9 — two bytes — because it's a UTF-8 encoded character outside the ASCII range.

When to encode

  • Query string values that may contain special characters (&, =, +, spaces)
  • Path segments containing non-ASCII characters
  • Form data submitted via GET requests

In JavaScript, use encodeURIComponent() for values and encodeURI() for full URLs. In Python, use urllib.parse.quote(). For quick encoding and decoding, the URL Encoder/Decoder handles both directions instantly.

// JavaScript
encodeURIComponent('hello world & more') // "hello%20world%20%26%20more"
decodeURIComponent('hello%20world')      // "hello world"

# Python
from urllib.parse import quote, unquote
quote('hello world & more')  # 'hello%20world%20%26%20more'

2. Base64 Encoding

Base64 converts binary data into a string of 64 printable ASCII characters (A–Z, a–z, 0–9, +, /). It was designed to solve a specific problem: some transmission channels (like SMTP email) were designed for text and would corrupt binary data. Base64 makes binary data safe to transmit as text.

What Base64 is NOT

Base64 is not encryption. It provides zero security — anyone can decode it instantly. Despite this, it's surprisingly common to see Base64 used to "hide" passwords or API keys in source code. Don't do this.

Common legitimate uses

  • Embedding images directly in HTML/CSS as data URIs (src="data:image/png;base64,...")
  • Encoding binary data in JSON APIs (JSON can't contain raw binary)
  • The payload section of JWT tokens
  • Email attachments (MIME encoding)
  • Basic Auth HTTP headers (Authorization: Basic dXNlcjpwYXNz)
# Python
import base64

encoded = base64.b64encode(b"hello world")  # b'aGVsbG8gd29ybGQ='
decoded = base64.b64decode(b'aGVsbG8gd29ybGQ=')  # b'hello world'

# JavaScript (browser)
btoa("hello world")        // "aGVsbG8gd29ybGQ="
atob("aGVsbG8gd29ybGQ=")  // "hello world"

Use the Base64 Encoder/Decoder for quick conversions without writing code.

3. HTML Encoding

HTML has reserved characters — <, >, &, ", ' — that the browser interprets as markup. If user-supplied content contains these characters and you output them unencoded, you have an XSS vulnerability.

<!-- User input: <script>alert('xss')</script> -->

<!-- Wrong -- executes the script: -->
<p><script>alert('xss')</script></p>

<!-- Correct -- displays the text safely: -->
<p>&lt;script&gt;alert('xss')&lt;/script&gt;</p>

Every modern templating engine (React, Vue, Django templates, Jinja2, Blade) HTML-encodes output by default. The vulnerability arises when developers use raw output methods (dangerouslySetInnerHTML, v-html, | safe) without sanitizing the input first.

4. Unicode and UTF-8

Unicode is a universal character standard covering over 140,000 characters across all human writing systems. UTF-8 is the most common encoding for Unicode on the web — it's variable-width, using 1 byte for ASCII characters and 2–4 bytes for others.

Always specify UTF-8 in your HTML and ensure your database connections use UTF-8 to avoid the classic "mojibake" problem (garbled characters like é appearing instead of é):

<!-- In HTML -->
<meta charset="UTF-8">

-- In MySQL
ALTER DATABASE mydb CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

Choosing the Right Encoding

A simple decision framework:

  • Putting data in a URL? → URL encode it with encodeURIComponent()
  • Base64 Encoder? → Base64
  • Outputting user content in HTML? → HTML Entity Encoder (use your framework's default escaping)
  • Storing or transmitting text? → UTF-8

For quick encoding tasks during development, QuickWebTools has dedicated tools for URL encoding and Base64 — no installation, no setup, just paste and go.