Know your nodes

Introduction

The parsermd package parses R Markdown and Quarto documents into an Abstract Syntax Tree (AST) representation. This vignette introduces the different types of AST nodes and their properties, helping you understand how parsermd represents document structure.

AST Container - `rmd_ast`

The rmd_ast object serves as the container for all parsed document nodes. It holds a linear sequence of nodes representing different document elements, where each node type corresponds to a specific R Markdown or Quarto construct (headings, code chunks, text, etc.).

Important: The AST represents documents as a linear sequence of nodes, not a nested tree structure. This means that structural elements like fenced divs are represented as separate opening and closing nodes in the sequence, rather than as nodes with children.

The default print method for rmd_ast’s (flat = FALSE) presents an implicit tree structure based on heading levels. This provides a hierarchical view that reflects the document’s logical organization, where content is grouped under headings based on their level.

Properties:

nodes: A list containing all the parsed nodes in document order

Example:

Raw text that would be parsed:

---
title: "Example Document"
---

# Introduction

This is some text.

```{r}
x <- 1:5
mean(x)
```

This would create an rmd_ast object containing:

rmd_yaml node with the title
rmd_heading node with “Introduction”
rmd_markdown node with “This is some text.”
rmd_chunk node with the R code

Programmatic creation:

ast = rmd_ast(list(
  rmd_yaml(list(title = "Example Document")),
  rmd_heading(name = "Introduction", level = 1L),
  rmd_markdown(lines = "This is some text."),
  rmd_chunk(
    engine = "r",
    code = c("x <- 1:5", "mean(x)")
  )
))

Hierarchical view (flat = FALSE):

print(ast, flat = FALSE)
#> ├── YAML [1 field]
#> └── Heading [h1] - Introduction
#>     ├── Markdown [1 line]
#>     └── Chunk [r, 2 lines] -

Linear view (flat = TRUE):

print(ast, flat = TRUE)
#> ├── YAML [1 field]
#> ├── Heading [h1] - Introduction
#> ├── Markdown [1 line]
#> └── Chunk [r, 2 lines] -

S7 Class System

parsermd uses the S7 object system for all AST node types. S7 provides a modern, robust class system with:

Type safety: Properties are validated when objects are created or modified
Performance: Efficient method dispatch and memory usage
Consistency: Uniform interface across all node types

Key S7 Features in parsermd:

All node types inherit from the base rmd_node class
Properties are accessed using @ syntax (e.g., node@content)
Validation ensures data integrity (proper types, lengths, etc.)
Method dispatch works seamlessly with generic functions

Property Access:

# Create a heading node
heading = rmd_heading(name = "Section Title", level = 2L)

# Access properties with @
heading@name
#> [1] "Section Title"
heading@level
#> [1] 2

Core Node Types

Document Structure Nodes

YAML Header - `rmd_yaml`

The rmd_yaml node represents YAML front matter at the beginning of documents.

Properties:

yaml: List containing the parsed YAML content

Example:

Raw text that would be parsed:

---
title: "My Document"
author: "John Doe"
date: "2023-01-01"
---

Programmatic creation:

yaml_node = rmd_yaml(list(
  title = "My Document",
  author = "John Doe",
  date = "2023-01-01"
))
yaml_node
#> <rmd_yaml>
#>  @ yaml:List of 3
#>  .. $ title : chr "My Document"
#>  .. $ author: chr "John Doe"
#>  .. $ date  : chr "2023-01-01"

Markdown Headings - `rmd_heading`

The rmd_heading node represents section headings in markdown.

Properties:

name: Character string containing the heading text
level: Integer from 1-6 indicating the heading level (# = 1, ## = 2, etc.)

Example:

Raw text that would be parsed:

# Introduction

Programmatic creation:

heading_node = rmd_heading(
  name = "Introduction", 
  level = 1L
)
heading_node
#> <rmd_heading>
#>  @ name : chr "Introduction"
#>  @ level: int 1

Markdown Text - `rmd_markdown`

The rmd_markdown node represents plain markdown text content.

Properties:

lines: Character vector containing the markdown text lines

Example:

Raw text that would be parsed:

This is a paragraph.
With multiple lines.

Programmatic creation:

markdown_node = rmd_markdown(
  lines = c("This is a paragraph.", "With multiple lines.")
)
markdown_node
#> <rmd_markdown>
#>  @ lines: chr [1:2] "This is a paragraph." "With multiple lines."

Code and Execution Nodes

Executable Code Chunks - `rmd_chunk`

The rmd_chunk node represents executable code chunks with options and metadata.

Properties:

engine: The code engine (default: “r”)
label: Optional chunk name/label
options: List of chunk options containing both traditional and YAML options
code: Character vector containing the code lines
indent: Indentation string
n_ticks: Number of backticks used (default: 3)

Chunk Option Formats:

Chunks support two option formats that can be used independently or together:

Traditional format: Options specified in the chunk header after the engine and label ```{{r chunk-label, eval=TRUE, echo=FALSE}}
YAML format: Options specified as YAML comments within the chunk
```
```{r chunk-label}
#| eval: true
#| echo: false
```
```

Option Conflict Resolution:

When the same option is specified in both formats, YAML options take precedence over traditional options. A warning is emitted when conflicts occur:

{r eval=TRUE} #| eval: false

In this case, eval: false (YAML) wins over eval=TRUE (traditional), and the parser emits: “YAML options override traditional options for: eval”

Type Handling:

Traditional options: Always stored as strings (e.g., "TRUE", "5")
YAML options: Preserve proper R types (e.g., TRUE, 5L, 3.14)

Examples:

Traditional format chunk:

```{r example, eval=TRUE, echo=FALSE}
x <- 1:10
mean(x)
```

YAML format chunk:

```{r example}
#| eval: true
#| echo: false
x <- 1:10
mean(x)
```

Mixed format chunk (with conflict):

```{r example, eval=TRUE}
#| eval: false
#| message: false
x <- 1:10
mean(x)
```

In this case, eval: false (YAML) overrides eval=TRUE (traditional).

Programmatic creation:

# Traditional-style options
chunk_node_traditional = rmd_chunk(
  engine = "r",
  label = "example",
  options = list(eval = "TRUE", echo = "FALSE"),
  code = c("x <- 1:10", "mean(x)")
)

# YAML-style options with proper types
chunk_node_yaml = rmd_chunk(
  engine = "r",
  label = "example",
  options = list(eval = TRUE, echo = FALSE),
  code = c("x <- 1:10", "mean(x)")
)

chunk_node_yaml
#> <rmd_chunk>
#>  @ engine : chr "r"
#>  @ label  : chr "example"
#>  @ options:List of 2
#>  .. $ eval: logi TRUE
#>  .. $ echo: logi FALSE
#>  @ code   : chr [1:2] "x <- 1:10" "mean(x)"
#>  @ indent : chr ""
#>  @ n_ticks: int 3

Raw Output Chunks - `rmd_raw_chunk`

The rmd_raw_chunk node represents raw output chunks for specific formats.

Properties:

format: The output format (e.g., “html”, “latex”)
code: Character vector containing the raw content
indent: Indentation string
n_ticks: Number of backticks used

Example:

Raw text that would be parsed:

```{=html}
<div class='custom'>
  <p>Custom HTML content</p>
</div>
```

Programmatic creation:

raw_chunk_node = rmd_raw_chunk(
  format = "html",
  code = c(
    "<div class='custom'>", 
    "  <p>Custom HTML content</p>", 
    "</div>"
  )
)
raw_chunk_node
#> <rmd_raw_chunk>
#>  @ format : chr "html"
#>  @ code   : chr [1:3] "<div class='custom'>" "  <p>Custom HTML content</p>" ...
#>  @ indent : chr ""
#>  @ n_ticks: int 3

Fenced Code Blocks - `rmd_code_block`

The rmd_code_block node represents non-executable fenced code blocks.

Properties:

id: Optional HTML ID attribute
classes: Character vector of CSS classes (e.g., language names like “python”, “r”)
attr: Named character vector for key-value attributes (e.g., c(style="color:blue"))
code: Character vector containing the code lines
indent: Indentation string
n_ticks: Number of backticks used

Example:

Raw text that would be parsed:

```python
def hello():
    print('Hello, World!')
```

Programmatic creation:

code_block_node = rmd_code_block(
  classes = c("python"),
  code = c(
    "def hello():", 
    "    print('Hello, World!')"
  )
)
code_block_node
#> <rmd_code_block>
#>  @ id     : chr(0) 
#>  @ classes: chr "python"
#>  @ attr   : chr(0) 
#>  @ code   : chr [1:2] "def hello():" "    print('Hello, World!')"
#>  @ indent : chr ""
#>  @ n_ticks: int 3

Code Block Literals - `rmd_code_block_literal`

The rmd_code_block_literal node represents code blocks with literal attribute capture using the {...} syntax. This format preserves the raw attribute content exactly as written, making it ideal for displaying code chunk examples.

Properties:

attr: Raw attribute string (exactly as written between {{ and }})
code: Character vector containing the code lines
indent: Indentation string
n_ticks: Number of backticks used

Example:

Raw text that would be parsed: {r, echo=TRUE, eval=FALSE} x <- 1:10 mean(x)

Programmatic creation:

code_block_literal_node = rmd_code_block_literal(
  attr = "r, echo=TRUE, eval=FALSE",
  code = c(
    "x <- 1:10", 
    "mean(x)"
  )
)
code_block_literal_node
#> <rmd_code_block_literal>
#>  @ attr   : chr "r, echo=TRUE, eval=FALSE"
#>  @ code   : chr [1:2] "x <- 1:10" "mean(x)"
#>  @ indent : chr ""
#>  @ n_ticks: int 3

Nested Braces Support:

The literal format can handle nested braces in attributes: {{r, code='function() { return(1) }'}}

This captures the attribute as: "r, code='function() { return(1) }'"

Structural Elements

Fenced Divs - `rmd_fenced_div_open` & `rmd_fenced_div_close`

Fenced divs are represented as pairs of nodes in the linear AST sequence. The rmd_fenced_div_open node marks the beginning of a fenced div block, and the rmd_fenced_div_close node marks the end. Any content between these nodes is considered to be inside the fenced div.

rmd_fenced_div_open Properties:

id: Optional HTML ID attribute
classes: Character vector of CSS classes
attr: Named character vector for key-value attributes

rmd_fenced_div_close Properties: None (just a marker)

Example:

Raw text that would be parsed:

::: {.warning #important}
This content is inside the fenced div.

More content here.
:::

This would create a sequence of nodes: 1. rmd_fenced_div_open with attributes 2. rmd_markdown with “This content is inside the fenced div.” 3. rmd_markdown with “More content here.” 4. rmd_fenced_div_close

Programmatic creation:

# Create the opening node
fenced_div_open_node = rmd_fenced_div_open(
  classes = c(".warning"),
  attr = c(id = "important")
)

# Create the closing node
fenced_div_close_node = rmd_fenced_div_close()

# These would typically be combined with content nodes in an rmd_ast
ast_with_div = rmd_ast(list(
  fenced_div_open_node,
  rmd_markdown(
    lines = "This content is inside the fenced div."
  ),
  rmd_markdown(
    lines = "More content here."
  ),
  fenced_div_close_node
))

Extracted Elements

The following classes represent elements that can be extracted from AST nodes through secondary parsing, rather than being direct nodes in the AST structure. These elements are found within markdown text and code content.

Inline Code - `rmd_inline_code`

The rmd_inline_code class represents inline code expressions extracted from markdown text.

Properties:

engine: The code engine (empty string for static code)
code: The inline code content
braced: Whether the code uses braced syntax
start: Starting position in the source text
length: Length of the inline code

Example:

Raw text containing inline code:

The result is 4.

Programmatic creation:

# Create directly
inline_code_obj = rmd_inline_code(
  engine = "r",
  code = "2 + 2",
  braced = FALSE
)
inline_code_obj
#>  rmd_inline_code[-1,-1] `r 2 + 2`

Shortcode Function Calls - `rmd_shortcode`

The rmd_shortcode class represents Quarto shortcode function calls extracted from markdown content.

Properties:

func: The shortcode function name
args: Character vector of arguments
start: Starting position in the source text
length: Length of the shortcode

Example:

Raw text containing a shortcode:

{{< embed type=video src=example.mp4 >}}

Programmatic creation:

# Create directly
shortcode_obj = rmd_shortcode(
  func = "embed",
  args = c("type=video", "src=example.mp4")
)
shortcode_obj
#>  rmd_shortcode[-1,-1] {{< embed type=video src=example.mp4 >}}

Spans - `rmd_span`

The rmd_span class represents inline span elements with attributes extracted from markdown text.

Properties:

text: The text content of the span
id: Optional HTML ID (must start with ‘#’ if present)
classes: Character vector of CSS classes (must start with ‘.’ if present)
attr: Named character vector of additional attributes

Example:

Raw text containing a span:

[Important text]{.highlight #key}

Programmatic creation:

# Create directly
span_obj = rmd_span(
  text = "Important text",
  id = c("#key"),
  classes = c(".highlight")
)
span_obj
#>  rmd_span [Important text]{#key .highlight}

Extraction Functions

These utility functions extract the above elements from AST nodes:

rmd_extract_inline_code(): Extract inline code from text
rmd_extract_shortcodes(): Extract shortcodes from text
rmd_extract_spans(): Extract spans from text
rmd_has_inline_code(): Check if text contains inline code
rmd_has_shortcode(): Check if text contains shortcodes
rmd_has_span(): Check if text contains spans

Introduction

AST Container - rmd_ast

S7 Class System

Core Node Types

Document Structure Nodes

YAML Header - rmd_yaml

Markdown Headings - rmd_heading

Markdown Text - rmd_markdown

Code and Execution Nodes

Executable Code Chunks - rmd_chunk

Raw Output Chunks - rmd_raw_chunk

Fenced Code Blocks - rmd_code_block

Code Block Literals - rmd_code_block_literal

Structural Elements

Fenced Divs - rmd_fenced_div_open & rmd_fenced_div_close

Extracted Elements

Inline Code - rmd_inline_code

Shortcode Function Calls - rmd_shortcode

Spans - rmd_span

Extraction Functions

AST Container - `rmd_ast`

YAML Header - `rmd_yaml`

Markdown Headings - `rmd_heading`

Markdown Text - `rmd_markdown`

Executable Code Chunks - `rmd_chunk`

Raw Output Chunks - `rmd_raw_chunk`

Fenced Code Blocks - `rmd_code_block`

Code Block Literals - `rmd_code_block_literal`

Fenced Divs - `rmd_fenced_div_open` & `rmd_fenced_div_close`

Inline Code - `rmd_inline_code`

Shortcode Function Calls - `rmd_shortcode`

Spans - `rmd_span`