Concepts

AI and LLM

AI and LLM

Tuning LLMDigestRules.yaml

What Each Rule Set Field Means

Each rule set in the file defines:

userRuleSets:

- name: my-site

domainPatterns:

- mysite.com

junkSelectors:

- "//aside"

- "//nav"

mainContentCandidates:

- "//article"

- "//main"

sitemapBaseUrl: "https://www.mysite.com"

enabledPasses:

- CollapseExcessiveBlanks

- HeadingSpacing

WARNING

XPath contains() is a substring test, so contains(@class,'ad') also matches shadow, gradient, leading and padding. On a site built from utility classes that strips the page. Match the whole class instead: contains(concat(' ', normalize-space(@class), ' '), ' ad ').

The Sixteen Cleanup Steps

Sixteen cleanup steps are available after the HTML-to-markdown conversion. Each rule set lists only the steps it wants by name, and a step not listed is skipped.

Whitespace and structure: CollapseExcessiveBlanks, HeadingSpacing, DedupeSections, DropEmptyHeadings, StripEscapeArtifacts.

Noise: RemoveCountersAndEmptyIcons, RemoveEmptyImages, StripMediaChrome.

Images: ImageGridBreaker, ImageSpacing, RejoinLinkedImages, DedupeConsecutiveImages.

Product pages: ProductImageBrandPrice, PriceGluing.

Links: SplitConsecutiveLinks puts one link per line instead of a row of cards running together, and ResolveRelativeLinks gives every root-relative target the page's own origin so the digest reads on its own.

Adding a Site-Specific Rule Set

A rule set only needs to define what makes a site different from the generic defaults. Put yours under userRuleSets, where they are yours and GPAL leaves them alone. To change a built-in rule, cut that whole entry out of ruleSets and paste it into userRuleSets, then edit it there: GPAL uses your copy for that name.

userRuleSets:

- name: linkedin

domainPatterns:

- linkedin.com

junkSelectors:

- "//aside[contains(@class,'sidebar')]"

- "//*[contains(@class,'social-action')]"

mainContentCandidates:

- "//main"

- "//*[contains(@class,'update-v2')]"

NOTE

If a field is missing from a rule set, GPAL fills it in from the generic rule set. So a site-specific entry can be as minimal as a name, domainPatterns, and the one or two fields that actually differ from generic.

On this page

AI and LLM
Tuning LLMDigestRules.yaml
  • What Each Rule Set Field Means
  • The Sixteen Cleanup Steps
  • Adding a Site-Specific Rule Set
    • 💬 Ask GPAL