YAML (YAML Ain't Markup Language) is a human-friendly, cross-language data serialization language that is commonly used for configuration files and represents data as mappings, sequences, and scalars. Version 1.2 made YAML a strict superset of JSON. In proxy and scraping work, YAML shows up in configuration files. For example, the Envoy proxy's static configuration is written in YAML and defines the listeners and clusters that traffic flows through.

YAML Syntax Basics

  • Mappings: key: value pairs, like dictionaries or hashes.
  • Sequences: Lists, with each item on its own line starting with -.
  • Indentation: In block style, indentation determines structure, so a misplaced space changes what the data means.
  • No tabs: Tab characters must not be used for indentation.
  • Comments: A # starts a comment.

Using YAML Safely in Scraper and Proxy Configs

YAML parsers can do more than read plain data. The PyYAML documentation warns that calling yaml.load on data from an untrusted source is not safe, because it can construct arbitrary Python objects. Its yaml.safe_load function recognizes only standard YAML tags, which makes it the documented choice for untrusted input.

Multi-line Strings, Anchors and Multiple Documents

  • Literal block (|): Keeps every line break in a multi-line string.
  • Folded block (>): Turns each line break into a space, except before empty or more-indented lines.
  • Anchors and aliases: &name marks a node, and *name reuses it later instead of repeating it.
  • Multiple documents: --- marks the start of a document's content, so one stream can hold several documents. ... ends a document without starting a new one.