Package: parser

errUnsupported is returned by the scanner for input outside the subset itcan parse exactly: nested maps, multi-line scalars, duplicate keys, andscalars yaml.v3 would type differently (floats, timestamps, hex, bigintegers). decodeAttrs then re-parses the whole block with yaml.v3, so theresult never depends on which path produced it.

decodeAttrs parses the contents of an ```attr block into a map. It runs thefast line scanner first and falls back to yaml.v3 for anything the scannercannot reproduce byte-for-byte (see errUnsupported).

decodeAttrYAML is the fast path: it parses a small flat YAML map (thecontents of an ```attr block) into map[string]any without building theintermediate YAML document AST that yaml.v3 allocates.

Supported syntax (covers all real-world attr blocks)🔑 value (scalar: string, int, bool)key: "quoted value" (quoted string — quotes stripped)key: (list — following lines are " - item")item1item2comment (ignored)blank lines (ignored)

Values are typed as yaml.v3 would type them:unquoted → string (unless it's an int or bool literal)"true"/"false" → boolinteger → intquoted → string (always, even if contents look like a number)list items → []any of strings

Anything it cannot reproduce exactly — malformed input, nested maps, flowmappings, duplicate keys, or a scalar yaml.v3 would retype (float,timestamp, hex, big integer) — returns errUnsupported so decodeAttrsre-parses the block with yaml.v3. Callers should use decodeAttrs, not thisfunction directly.

TestDecodeAttrs_DuplicateKey rejects a repeated key instead of silentlykeeping the last value (yaml.v3 rejects it too, with a line number).

TestDecodeAttrYAML_UnsupportedScalarIsReported guards the fast path: ascalar it cannot type must be reported as errUnsupported so decodeAttrsfalls back, never silently stringified.

TestParseScalar covers the plain-scalar typing the scanner must reproduceexactly; anything it cannot type is rejected so the block re-parses withyaml.v3 instead of silently becoming a string.

TestUnquoteYAML covers both quote styles: doubled single quotes are aliteral quote, double quotes carry YAML escapes, and an unterminated quoteis an error (which send the block to yaml.v3).

TestParseFlowSeq covers flow sequences, including quoted items with commas(which a naive string split would break) and invalid input.

TestSplitFlowItems covers the quote- and bracket-aware comma split.

parseScalar converts an unquoted YAML plain scalar to the Go type yaml.v3would produce: true/false → bool, null/~ → nil, integer → int, anythingelse → string. Scalars yaml.v3 would retype as something this functioncannot produce (float, timestamp, hex, big integer) report errUnsupportedso decodeAttrs re-parses the block with yaml.v3 instead of silentlyturning them into strings.

isPlainStringStart reports whether c can begin a plain YAML scalar thatyaml.v3 resolves as a string.

isIntegerString reports whether s is a plain integer literal (optionalsign followed by digits). Used as the gate before strconv.Atoi.

unquoteYAML strips single or double quotes from a YAML scalar.

parseFlowSeq parses a YAML flow sequence [item1, item2, item3] into []any.Items are parsed as scalars (same rules as parseScalar). Quoted itemsare supported.

splitFlowItems splits a flow sequence body on commas, respecting quotes.

benchAttr is a representative minimal attr block: two scalars, one int, onelist with pins. Both benchmarks decode it so the scanner's advantage staysmeasurable — it is ~15x faster than yaml.v3 (1784 vs 27608 ns/op, 14 vs 140allocs) and that is why the fast path exists at all. If the gap ever closes,delete the scanner and keep yaml.v3.

BenchmarkParseMD measures the whole file→content-tree path, which is whatreqmd check/ls spend nearly all their time in.

syntheticDoc builds a document of roughly n bytes made of root-levelrequirements with attr blocks — the shape that exercises both theroot-split pre-scan (above parseParallelThreshold) and the chunkedparallel parse.

DiscoverAtTagWithSchemas extracts the repo at the given git tag into a tempdirectory, runs the standard Discover pipeline, and returns the parseddocuments plus the raw schema.yaml content for each document directory.

discoverAtTag extracts the repo at the given git tag and returns both theparsed documents and the raw schema maps per document path.

rewriteNodeSources applies fn to the Source of every node in the tree.

relToTmpDir returns path relative to the temp extraction directory.

ListSubmodulesAtTag returns a map of submodule path → pinned commit SHAat the given git tag. Uses git ls-tree -t <tag> and filters forsubmodule entries (mode 160000, type commit). Returns an empty mapif the tag has no submodules. Returns an error if the tag is invalidor the git command fails.

parseLsTreeOutput parses the output of git ls-tree -t <tag> andreturns only submodule entries (mode 160000, type commit) aspath → SHA. Exposed as a separate function for unit testability.

git ls-tree -t output format (one entry per line): \t

Example:160000 commit abc123def456...\tvendor/spec-a 100644 blob 789ghi012jkl...\tREADME.md

mdParser is the shared goldmark instance used by parseMD.Constructed once at package init to avoid re-running extension Init for every .md file.

Discover walks root recursively, locating schema.yaml files.Every directory containing schema.yaml is a document directory.Returns one Document per document directory.Document directories are loaded in parallel for performance.

findDocDirs walks root and collects all directories containing schema.yaml.

LoadSchema reads and parses the schema.yaml file from a document directory.It returns the raw schema map before any x-reqmd extraction.

parseFiles parses all .md files in a document directory and returns theconcatenated content tree roots in mdFiles order (deterministic), thedocument title (the first file's h1 heading, if any), mergedfrontmatter, and any per-file errors. Files are parsed in parallel;results are collected by file index so the returned order never dependson goroutine completion order.

parseParallelThreshold is the minimum file size (in bytes) for which theroot-split pre-scan + intra-file chunk parallelism is worth it. Belowthis size the file is parsed serially: the pre-scan is an extra fullline scan plus a frontmatter YAML parse, and a small file parses inwell under a millisecond so chunking cannot help. Cross-file parallelism(the worker pool in parseFiles) still applies.

parseMD parses a single .md file and returns the assembled content treeroots, the file title (the first h1 heading, when present), and any YAMLfrontmatter as a map.

Level-1 headings are document titles, not content: they are never nodesand can never be requirements (only h2+ can). The first h1 in the file iscaptured as the file title; the tree assembled from the remaining nodesstarts at the content heading level.

For large files with multiple root-level requirements (parent=nil), thefile is split at root requirement boundaries and chunks are parsed inparallel. Each chunk yields a flat node stream in source order; thestreams are concatenated and assembled into the tree afterwards sonesting across chunk boundaries (e.g. a container in chunk 0 with arequirement in chunk 1) is preserved. The pre-scan to find splitpoints is a cheap line scan that replicates the stack logic withoutbuilding a goldmark AST.

Compute chunk boundaries: which split indices start/end each chunk.Chunk 0 additionally covers the file preamble [0, splits[0]) soleading containers and prose (including the h1 title) are neverdropped.

parseMDChunk parses a single chunk of markdown (possibly the whole file)using goldmark and returns the flat node stream in source order, the filetitle (first h1 heading), and any YAML frontmatter. Every heading excepta level-1 document title becomes a node; prose and other blocks attach tothe most recently opened node's Body. Nodes are NOT assembled into a treehere — callers must assemble the concatenated streams so nesting acrosschunk boundaries is preserved.

assembleTree builds the document content tree from a flat, ordered nodestream. A node's parent is the nearest preceding heading with ashallower level; requirements additionally get ParentID pointing at thenearest preceding requirement heading. Info nodes with children arepromoted to KindContainer.

popByLevel removes stack entries whose level is >= target, mirroringthe heading-nesting rule: a heading at level L closes everything atlevel L or deeper.

promoteContainers upgrades info nodes that have children to containers.

reqStackEntry tracks an open requirement at a heading level during theroot-split pre-scan.

popStackToLevel removes entries whose level is >= targetLevel.Used by the root-split pre-scan to replicate requirement nesting.

findRootSplits scans src line-by-line to find byte offsets where root-levelrequirements begin. A root requirement is one where the requirement stackis empty when the requirement heading is encountered. The scan replicatesthe same stack logic as the assembly step but without building a goldmarkAST.

Returns the byte offsets of root requirement starts (including the headingline) and any YAML frontmatter parsed from the file header.

If only 0 or 1 roots are found, the caller parses the whole file serially.

nextLine returns the line at byte offset pos (without the trailing \n)and whether there was a newline. The returned slice references src —no allocation.

headingLevelBytes returns the heading level (1-6) and true if the lineis a markdown heading (e.g. "## Title"). Returns 0, false otherwise.Works on []byte to avoid string allocation.

nextNonBlankIsAttrBytes checks if the next non-blank line after byteoffset pos is a ```attr fence. Scans src directly — no allocation.

isNextAttrBlock returns true if n's next sibling is a ```attr fence.Blank lines between heading and fence are consumed by goldmark and don'tcreate AST nodes, so they don't affect this check.

splitHeadingID splits a heading like "REQ-001: This is a title"into ("REQ-001", "This is a title"). If there's no colon+space,returns (heading, "").

--- text extraction helpers ---

ParseSingleFile is used by reqmd check <file> --schema <schema>.

parseReqTree is a test helper: parses src and returns the assembledcontent tree roots.

reqsOf returns the flat requirement list of an assembled tree.

TestParseMD_Full verifies a normal requirement with heading, attr block,prose body, and *Rationale: line.

TestParseMD_NoRationale verifies that a requirement without *Rationale:*has an empty Rational field.

TestParseMD_MultipleRequirements verifies that multiple requirementsin one document are each parsed correctly.

TestParseMD_HeadingWithoutAttrIsInfo verifies that a heading without anattr block produces an info item instead of a requirement.

TestParseMD_BadYAML verifies that invalid YAML inside an attr blockreturns an error.

TestParseMD_NonAttrCodeBlock verifies that a fenced code block with anon-attr info string (e.g. ```json) inside a requirement body is preserved.

TestParseMD_EmptyFile verifies that parsing an empty source returnsno nodes and no error.

TestParseMD_ThematicBreak verifies that thematic breaks (---) insidea requirement body are preserved and don't disrupt body accumulation.

TestParseMD_ArrayAttr verifies that an attribute with a YAML array valueis parsed into []interface{} with the correct elements.

TestParseMD_SubReqUnderParent verifies that a ### heading with an attr blockimmediately following a ## heading creates a sub-requirement with ParentID set.

TestParseMD_MultipleSubReqs verifies that multiple ### sub-requirementsunder the same ## parent all get the correct ParentID.

TestParseMD_SubReqDynamicLevel verifies that a ### heading with attr blockas the FIRST requirement is a top-level requirement, not an orphan child.

TestParseMD_SubHeadingBecomesInfo verifies that a ### heading without anattr block under a requirement becomes an info item child, not body text.

TestParseMD_DynamicReqLevel verifies that requirements chain by headinglevel when the content tree starts at level 2 (h1 is reserved for thedocument title).

TestParseMD_H1IsDocumentTitle verifies that a level-1 heading is treatedas the document title: an h1 with an attr block is rejected (requirementsmust be level 2+), and a plain h1 is captured as the title and neverbecomes a content node.

TestParseSingleFile verifies ParseSingleFile reads a real file from diskand produces the correct results. Uses t.TempDir() for the temp file.

TestParseMD_Title verifies that a heading with "ID: Title" formatsplits the ID and Title correctly.

TestParseMD_NoTitle verifies that a heading without ": " has empty Title.

TestParseMD_TitleWithColonInTitle verifies that only the first ": " splits(IDs can't contain colons per the ID pattern, but titles may).

TestParseMD_ContainerHoldsRequirements verifies that a container heading(no attr block) becomes a KindContainer node that holds the requirementsnested under it, which stay top-level (no requirement parent).

TestParseMD_UserExample1 reproduces the scenario the user described:a container at the same level breaks the requirement chain; requirementsnest under containers and stay top-level.

TestParseMD_DeepNesting verifies requirements at arbitrary depth(3+ levels) chain correctly when no containers intervene.

TestParseMD_ContainerDoesNotBreakDeeperChain verifies that a containerheading deeper than the current requirement does NOT break the chain:requirements after it still link to the nearest shallower requirement.

TestParseMD_LevelGapNoContainer verifies that a heading gap without acontainer still links children to the nearest shallower requirement.

TestParseMD_SiblingContainersBreakChain verifies that multiple containersat successive levels produce only top-level requirements.

TestParseMD_Preamble verifies that leading prose before the first headingis captured as an info node and not dropped, and that an h1 title does notbecome a wrapping container.

TestParseMD_RequirementlessFile verifies that a file with no requirementheadings at all is captured entirely as info nodes, with the h1 title keptout of the content tree.

TestParseMD_ChunkedPreamble verifies that the parallel chunked path(multiple root requirements) still captures the file preamble and title.

TestParseMD_LevelGapUnderContainer verifies that a requirement inside acontainer under another requirement gets the right ParentID (nearestshallower requirement) and container nesting.