encoding · level 9

PEM & ASN.1

Why every TLS/PGP/SSH key file looks similar but isn't.

200 XP

PEM & ASN.1

Open any TLS certificate, any SSH key file, any PGP private key, and you'll see the same shape:

-----BEGIN CERTIFICATE-----
MIIDazCCAlOgAwIBAgIUSQVjSp8mY7nvHjKP4Qd9WEy1Sx4wDQYJKoZIhvcNAQEL
...
-----END CERTIFICATE-----

Different label, same skeleton. They look interchangeable — they aren't. Understanding the layered format that makes them all look similar is the difference between confidently inspecting a key and trial-and-erroring against openssl until something works.

The three-layer stack

PEM   = base64(DER) wrapped in BEGIN/END markers
DER   = canonical binary encoding of an ASN.1 value
ASN.1 = the schema (what fields, what types, in what order)

Top-down, three independent things that always travel together:

  • ASN.1 (Abstract Syntax Notation One) is a schema language from the 1980s. You write things like Certificate ::= SEQUENCE { tbsCertificate TBSCertificate, signatureAlgorithm AlgorithmIdentifier, signatureValue BIT STRING }. It says nothing about bytes — it's pure structure.
  • DER (Distinguished Encoding Rules) is one canonical way to serialise an ASN.1 value into bytes. Tag-Length-Value: each field starts with a one-byte tag (0x30 for SEQUENCE, 0x02 for INTEGER, 0x04 for OCTET STRING…), then a length, then the value bytes. "Distinguished" means there's exactly one valid encoding per value — important for signatures.
  • PEM (Privacy-Enhanced Mail) is text wrapping for DER. Base64-encode the DER bytes, line-wrap at 64 columns, sandwich between -----BEGIN <LABEL>----- and -----END <LABEL>-----. The label tells the reader what schema the inner DER follows.

That's the entire stack. Three layers, no magic.

The labels that matter

Label Schema What it is
CERTIFICATE X.509 A signed certificate. The thing servers send during TLS.
CERTIFICATE REQUEST PKCS#10 A CSR — a request to a CA. Contains a subject and a public key.
PRIVATE KEY PKCS#8 Algorithm-agnostic private key wrapper. Modern default.
RSA PRIVATE KEY PKCS#1 Legacy RSA-only private key format.
EC PRIVATE KEY SEC1 Legacy EC-only private key format.
PUBLIC KEY SPKI SubjectPublicKeyInfo — wraps any public key with an algorithm OID.
RSA PUBLIC KEY PKCS#1 Legacy RSA-only public key.
OPENSSH PRIVATE KEY OpenSSH Not PKCS#8 — OpenSSH's own format. ssh-keygen round-trips it.
ENCRYPTED PRIVATE KEY PKCS#8 PKCS#8 with a password-derived encryption layer.
DH PARAMETERS PKCS#3 Diffie-Hellman group parameters.

Same envelope, very different schemas inside. Mismatched label/schema = "unable to load key" errors.

The bytes themselves

Decode the base64 in any PEM file and the first byte is almost always 0x30 — ASN.1's "SEQUENCE" tag. Crypto objects start with a SEQUENCE because they're structured records.

$ openssl x509 -in cert.pem -outform DER | xxd | head -3
00000000: 3082 03b0 3082 0298 a003 0201 0202 0900  0...0...........
00000010: e6f5 9e88 7d2c 4e88 300d 0609 2a86 4886  ....},N.0...*.H.
00000020: f70d 0101 0b05 0030 5d31 0b30 0906 0355  .......0]1.0...U

30 82 03 b0 decodes as: tag SEQUENCE, length form "long, two bytes follow", length = 0x03b0 = 944 bytes. The next byte is another 30 because the outer SEQUENCE contains a TBSCertificate (also a SEQUENCE), and so on, recursively, all the way down to the OIDs and signatures at the leaves.

You don't need to read DER by hand. You do need to know the shape so that when an error message says "expected SEQUENCE got SET at offset 47", it doesn't read like alien.

Why openssl wants you to know

Almost every conversion you do in a CLI is moving up and down this stack:

# PEM → DER (just unwrap base64)
openssl x509 -in cert.pem -outform DER -out cert.der

# DER → PEM
openssl x509 -in cert.der -inform DER -out cert.pem

# Inspect — print the ASN.1 structure
openssl asn1parse -in cert.pem
openssl x509 -in cert.pem -text -noout

# Convert PKCS#1 → PKCS#8 (the modern default)
openssl pkcs8 -topk8 -nocrypt -in legacy-rsa.pem -out modern-pkcs8.pem

The same cert can exist in any of these three forms; the actual bytes and meaning don't change.

OIDs — the universal name system

Inside the ASN.1 tree you'll see things like 1.2.840.113549.1.1.11. These are Object Identifiers — globally-unique dotted-decimal names assigned by IANA, ISO, and various private branches. 1.2.840.113549.1.1.11 means "RSA encryption with SHA-256" (sha256WithRSAEncryption). 1.3.132.0.34 means "secp384r1". Cert chains, signature algorithms, key purposes, extension types — all OIDs.

You'll never memorise them. You will eventually learn the few you see weekly. oid-info.com is the reverse-lookup that lives in browser tabs.

Common foot-guns

  • Mismatched label and content. A PKCS#8 file with -----BEGIN RSA PRIVATE KEY----- will refuse to load — the parser tries to apply the PKCS#1 grammar and chokes. Fix the label, not the bytes.
  • Trailing whitespace and CRLF. Some parsers are strict — extra whitespace after END lines, or Windows line endings, cause "no end line" errors. dos2unix is your friend.
  • OpenSSH ≠ PKCS#8. -----BEGIN OPENSSH PRIVATE KEY----- is OpenSSH's own format. Convert with ssh-keygen -p -m PEM -f keyfile to get something openssl understands.
  • Multiple PEM blocks per file. A cert chain often concatenates CERTIFICATE, CERTIFICATE, CERTIFICATE. Most parsers handle this; some don't. When loading "fails for no reason", count the blocks.
  • DER is not PKCS#7 is not PFX. .cer and .crt files are usually PEM or DER X.509. .p7b is a PKCS#7 SignedData wrapper. .pfx / .p12 is PKCS#12 — a password-encrypted archive of certs and keys. Different beast.

The takeaway

Every key and cert you'll ever debug is base64(DER(ASN.1-of-something)). The label tells you what "something" is. Once that triangle is in your head, everything else — the openssl flags, the missing-newline errors, the Java keystores, the PEM-vs-PKCS conversions — becomes a routine traversal of three known layers.

Tools in the wild

4 tools
  • opensslfree tier

    The Swiss-army knife of PEM/DER. `openssl x509 -in cert.pem -text` prints any cert.

    cli
  • asn1js (web)free tier

    Paste a base64 PEM body and see the ASN.1 tree. Indispensable for debugging cert parsing.

    service
  • step-clifree tier

    Modern openssl alternative — `step certificate inspect` is friendlier.

    cli
  • Python's defacto X.509 library. Loads PEM/DER, parses extensions, builds CSRs.

    library