Advanced Configuration

This page covers advanced configuration options for the sphinx-llms-txt extension.

Customizing the LLMs Files

By default, the extension generates two files:

1. llms.txt - A summary file in Markdown format

2. llms-full.txt - A complete documentation file in reStructuredText format

You can customize these files in several ways:

Changing Filenames

You can change the default filenames by setting these values in your conf.py:

llms_txt_filename = "custom-summary.txt"
llms_txt_full_filename = "custom-docs.txt"

Disabling File Generation

If you only want one of the files, you can disable generation of the other:

# Disable summary file
llms_txt_file = False

# Disable full documentation file
llms_txt_full_file = False

Adding a Custom Summary

The summary file can include a custom description of your project:

llms_txt_summary = """
This documentation explains how to use MyProject to build amazing
applications. The project provides a comprehensive API for handling
data processing and visualization.
"""

Note

The summary can span multiple lines and will be properly formatted in the output file.

Custom Title

By default, the project name from Sphinx is used as the title in llms.txt. You can override this:

llms_txt_title = "My Custom Project Documentation"

Handling Large Documentation

For very large documentation sets, generating the full documentation file might exceed reasonable size limits. You can set a maximum line count and control what happens when that limit is exceeded:

llms_txt_full_max_size = 10000  # Maximum 10,000 lines
llms_txt_full_size_policy = "warn_skip"  # Default behavior

The llms_txt_full_size_policy setting controls both the log level and action taken when the size limit is exceeded. It uses the format "<loglevel>_<action>":

Log levels: - warn: Log as a warning (default) - info: Log as informational message

Actions: - skip: Don’t create the file (default) - keep: Create the file anyway, ignoring the size limit - note: Create a placeholder file explaining why the full file wasn’t generated

Tip

Use Excluding Content to remove less relevant pages and reduce the file size.

Custom Directive Handling

Path Resolution

The extension resolves paths in the common directives [ 'image', 'figure'] by default. You can add custom directives to this list:

llms_txt_directives = [
    "my-custom-image-directive",
    "another-directive-with-paths",
]

This ensures that paths in your custom directives are properly resolved in the generated files.

Excluding Content

There are several ways to exclude content from the generated llms-full.txt file:

Global Page Exclusion

You can exclude specific pages from being included in the generated files:

llms_txt_exclude = [
    "search",  # Exclude the search page
    "genindex",  # Exclude the index page
    "private_*",  # Exclude all pages starting with 'private_'
]

This is useful for excluding auto-generated pages, indexes, or content that isn’t relevant for LLM consumption. It can also be used to reduce the size of llms-full.txt.

Page-Level Ignore Metadata

You can exclude individual pages by adding metadata at the top of any reStructuredText file:

:llms-txt-ignore: true

Page Title
==========

This entire page will be excluded from llms-full.txt

When this metadata is present, the entire page is skipped during processing.

Block-Level Ignore Directives

You can exclude specific sections within a page using ignore directives:

Page Title
==========

This content will be included in llms-full.txt.

.. llms-txt-ignore-start

This content will be excluded from llms-full.txt.

Section To Ignore
-----------------

This entire section and any nested content will be ignored.

.. code-block:: python

   # This code block will also be ignored
   def ignored_function():
       pass

.. llms-txt-ignore-end

This content will be included again.

Block-level ignores can be useful for:

• Removing internal notes or TODOs

• Hiding implementation details while keeping user-facing documentation

Note

• Multiple ignore blocks can be used within the same file

• Ignore directives work with any indentation level

Including Source Code Files

You can include source code files from your project at the end of llms_txt_full_filename.

Use include/exclude syntax to precisely control which files are included:

llms_txt_code_files = [
    "+:src/**/*.py",           # Include all Python files in src
    "-:src/**/__pycache__/**", # Exclude Python cache files
]

Pattern syntax:

• +:pattern: Include files matching the pattern. Processed first to collect matching files.

• -:pattern: Exclude files matching the pattern. Applied to filter out unwanted files.

Code files are processed as follows:

• Glob patterns: Use standard glob patterns (*, **, ?) to match files

• Relative paths: Patterns are resolved relative to your Sphinx source directory

• Formatting: Each file is presented with a title and syntax-highlighted code block

Customizing Code File Paths

By default, the extension automatically detects the relative path from your Sphinx source directory to the git root and strips that prefix from displayed file paths. You can customize this behavior:

# Manually specify base path to strip
llms_txt_code_base_path = "../../"

# Disable path stripping entirely
llms_txt_code_base_path = ""

This helps create cleaner, more readable file paths in the generated documentation.

Using HTML Base URL

If you want to include absolute URLs for resources in your documentation, you can use Sphinx’s built-in html_baseurl configuration:

html_baseurl = "https://example.com/docs/"

When this option is set, all resolved paths in directives will be prefixed with this URL, creating absolute paths in the generated files.

Customizing URI Links in llms.txt

By default, the llms.txt file links to source files in the _sources directory when available, falling back to HTML pages when sources aren’t available. You can customize this behavior using URI templates with llms_txt_uri_template:

# Default: Link to source files, if _sources exists
llms_txt_uri_template = "{base_url}_sources/{docname}{suffix}{sourcelink_suffix}"

# Default: Link to HTML pages instead, if _sources doesn't exist
llms_txt_uri_template = "{base_url}{docname}.html"

# Manual: Link to a custom markdown build
llms_txt_uri_template = "{base_url}{docname}.md"

Available Template Variables

Your URI template can use the following variables:

• {base_url} - The base URL from html_baseurl configuration (includes trailing slash)

• {docname} - The document name (e.g., index, guide/intro)

• {suffix} - The source file suffix (e.g., .rst, .md) - may be empty if no source file exists

• {sourcelink_suffix} - The suffix from html_sourcelink_suffix configuration (e.g., .txt)

Tip

Instead of using the default of linking to _sources, you can generate Markdown and/or reStructuredText files from your documentation and link to those in llms.txt. See CMake Workflow for an example of building both HTML and Markdown and/or reStructuredText in parallel. Note that _sources is still needed for llms-full.txt at this time.

CMake Workflow

This project uses CMake to orchestrate documentation builds across multiple output formats, serving as a simple demo of the functionality. This approach enables parallel builds and integrates well with CI/CD platforms like Read the Docs.

Building multiple formats allows you to compare what works best for your docs, as well as allows users to choose which format to feed to their LLM. Use llms_txt_uri_template to configure links to point to your preferred format.

Key Files

These configuration files serve as a simple example of a Sphinx site hosted on Read The Docs, some modification may be needed.

.
├── .readthedocs.yml
├── CMakeLists.txt
├── CMakePresets.json
└── docs/
    └── CMakeLists.txt

Each section below contains a summary of the file’s purpose, the full contents of the file, and a table describing key lines that may need modification.

.readthedocs.yml

A Read The Docs config file that installs dependencies, then runs the full documentation workflow which builds all output formats in parallel, and copies them into a single deploy location.

version: 2

build:
  os: ubuntu-24.04
  tools:
    python: "3.13"
  commands:
    - pip install cmake
    - pip install -r docs/requirements.txt
    - cmake --workflow --preset documentation-workflow
    # Copy built documentation to Read the Docs output directory
    - mkdir -p $READTHEDOCS_OUTPUT/html
    - cp -r build/html/* $READTHEDOCS_OUTPUT/html/
    - cp -r build/markdown/* $READTHEDOCS_OUTPUT/html/
    - cp -r build/rst/* $READTHEDOCS_OUTPUT/html/

Line	Description
9	Update the path if your requirements file is in a different location
13-14	Modify the copy commands for the output formats you deploy

CMakeLists.txt

A CMake config file that sets up the project, fetches the shared sphinx-cmake-modules, and includes the docs subdirectory.

cmake_minimum_required(VERSION 3.15)
project(SphinxDocs VERSION 1.0.0 LANGUAGES NONE)

# Fetch Sphinx CMake modules
include(FetchContent)
FetchContent_Declare(
  sphinx_cmake_modules
  GIT_REPOSITORY https://github.com/jdillard/sphinx-cmake-modules.git
  GIT_TAG        main
)
FetchContent_MakeAvailable(sphinx_cmake_modules)
list(APPEND CMAKE_MODULE_PATH "${sphinx_cmake_modules_SOURCE_DIR}/cmake/modules")

# Add documentation
add_subdirectory(docs)

Line	Description
9	Update the GIT_TAG to use a different version or commit hash
15	Change if your docs subdirectory has a different location

docs/CMakeLists.txt

A CMake config file that includes the SphinxUtils module from FetchContent and defines the documentation-specific build targets.

include(SphinxUtils)

setup_sphinx_environment()

add_sphinx_builder(html)
add_sphinx_builder(markdown)
add_sphinx_builder(rst)

Line	Description
5-7	Add or remove calls based on which output formats you need

CMakePresets.json

Defines presets for configuring and building documentation:

• Configure Presets: Sets up the build directory.

• Build Presets: Defines build formats individually and all in parallel.

• Workflow Presets: Runs the configure preset followed by the parallel build preset.

{
  "version": 6,
  "configurePresets": [
    {
      "name": "documentation",
      "displayName": "Documentation Build",
      "description": "Configure project with documentation environment setup",
      "binaryDir": "${sourceDir}/build"
    }
  ],
  "buildPresets": [
    {
      "name": "html",
      "displayName": "Build HTML Documentation",
      "configurePreset": "documentation",
      "targets": ["html"]
    },
    {
      "name": "markdown",
      "displayName": "Build Markdown Documentation",
      "configurePreset": "documentation",
      "targets": ["markdown"]
    },
    {
      "name": "rst",
      "displayName": "Build reStructuredText Documentation",
      "configurePreset": "documentation",
      "targets": ["rst"]
    },
    {
      "name": "docs-parallel",
      "displayName": "Build all output formats in parallel",
      "configurePreset": "documentation",
      "targets": ["html", "markdown", "rst"]
    }
  ],
  "workflowPresets": [
    {
      "name": "documentation-workflow",
      "displayName": "Documentation Build Workflow",
      "steps": [
        {
          "type": "configure",
          "name": "documentation"
        },
        {
          "type": "build",
          "name": "docs-parallel"
        }
      ]
    }
  ]
}

Line	Description
18-23	Remove this preset to disable Markdown documentation builds
24-29	Remove this preset to disable reStructuredText documentation builds
34	Modify the targets list to build only the output formats you need in parallel

Usage

To build documentation locally using CMake:

# Run the full workflow (configure + build all formats)
cmake --workflow --preset documentation-workflow

# Or configure and build separately
cmake --preset documentation
cmake --build --preset html        # Build HTML only
cmake --build --preset docs-parallel  # Build all formats

Integration Examples

Complete Configuration Example

Here’s a complete example showing multiple Project Configuration Values:

# File names and generation options
llms_txt_filename = "ai-summary.txt"
llms_txt_full_filename = "ai-full-docs.txt"
llms_txt_full_max_size = 50000
llms_txt_full_size_policy = "warn_note"

# Content customization
llms_txt_title = "Project Documentation for AI Assistants"
llms_txt_summary = """
This is a comprehensive documentation set for our project.
It includes API references, usage examples, and tutorials.
"""
llms_txt_uri_template = "{base_url}{docname}.md"

# Path handling
html_baseurl = "https://docs.example.com/"
llms_txt_directives = ["custom-image", "custom-include"]

# Content filtering
llms_txt_exclude = ["search", "genindex", "404", "private_*"]

# Source code inclusion with include/exclude patterns
llms_txt_code_files = [
    "+:../../src/**/*.py",           # Include Python files
    "+:../../config/*.yaml",         # Include config files
    "-:../../src/**/__pycache__/**", # Exclude cache files
]
llms_txt_code_base_path = "../../"