Ingest a source tree

A source tree is the best thing you can give the knowledge pipeline to read. It is real code that uses the technology, so what the pipeline finds there is what people actually do with it. Fetched documentation is the fallback, for a technology that has no code for you to point at.

This page is for an author about to identify knowledge components from a project on their disk. It covers the --source flag and the bundle it builds. For documentation pages rather than code, see Fetch documentation from a URL.

There is no ingest command

Ingestion is a flag, not a step you run on its own. You pass --source <path> to a command that needs something to read, and that command bundles the tree before it does anything else. These commands take it:

• the three identify commands — see Identify knowledge components
• /fito-create-or-update-technology, which drafts a technology’s descriptions from what it reads — see Provision a technology

Before you begin

• A pi session in a working directory of your own, with Fito installed there. The bundle is written under that directory, and Fito refuses to write it inside its own package. See Set up a working directory.
• The project, on your disk. A clone is enough — the bundle reads the working tree and never the git history.
• For an identify command, the technology and its major line already provisioned in Fito.

Bundle a tree

1. In your pi session, pass --source to an identify command:

   /fito-identify-domain-concepts Javalin --technology javalin --technology-version 7 --source ~/code/my-javalin-app

   Replace ~/code/my-javalin-app with the path to your project. A relative path resolves against your working directory.

2. Let the command run. It bundles the tree first, then hands the bundle to the identifier as the thing to read. The flag wins over --fetch: when a bundle is built, the command does not fetch.

Confirm it worked: the bundler prints one line naming the count and the destination, and a third line when it left files out:

✓ Bundled <n> files (<bytes> bytes) from <source-path>
  into <working-dir>/output/1-ingest/source_bundle
  <k> skipped (binary / oversized / symlink) — see bundle.json

What the bundle holds

The bundle lands at ./output/1-ingest/source_bundle/, and it has two parts:

• files/ — a copy of every file that made it in, at the same relative path it had in your project. The identifier reads, greps and lists it as if it were the real tree.
• bundle.json — the source path, the time it was built, each bundled file with its size, and each skipped file with the reason it was skipped.

Five kinds of thing are left out:

Left out	Rule	Listed in bundle.json
Tooling directories	.git, .hg, .svn, node_modules, build, dist, target, coverage, .gradle, .idea, .vscode, .next, .svelte-kit, .venv, venv, __pycache__, the pipeline’s own output, and the case-authoring folders .case and .authoring	No
Lockfiles and noise	package-lock.json, yarn.lock, pnpm-lock.yaml, bun.lockb, .DS_Store	No
Binary files	A NUL byte in the first 8 KiB	Yes, as binary
Large files	Over 256 KiB	Yes, as too large (… bytes > 262144 cap)
Symbolic links	Never followed	Yes, as symlink (not followed)

A large central file is not read

The size cap applies to every file, including the one that matters most. A project whose core class is a single 300 KiB source file loses that file, and the identifier never sees it. Check bundle.json for the skip, and supplement with fetched documentation when the file you lost is the one that explains the technology.

Bundle again for the next command

Each identify command with --source rebuilds the bundle from scratch, and replaces the previous one whole. That is cheap, and it is the simplest way to run all three identify commands over the same tree: pass the same --source to each.

Leaving --source off does not reuse the bundle. Without it, an identify command reads --docs, and --docs defaults to the fetched-documentation folder, ./output/1-ingest/extracted_links_content/. To reuse a bundle without rebuilding it, point --docs at it:

/fito-identify-usage-scenarios Javalin --technology javalin --technology-version 7 --docs ./output/1-ingest/source_bundle/

Ingest a case you are authoring

When you work inside a case checkout — a directory holding .case/manifest.yaml — an identify command given none of --source, --docs or --fetch checks whether the checkout is yours to author. If it is, it offers to proceed with --source ., and you answer.

The check is a question about where the checkout came from:

• Authoring — the case repository itself, a tree with no origin remote, a non-GitHub origin, or a fork of a repository Fito has not registered.
• Not authoring — a GitHub fork of a published case registered in Fito. That is exactly what /fito-case-open makes for a learner, so no offer is made.

The check needs git, an authenticated gh and a reachable Fito server. When it cannot decide — gh missing, the server down — it makes no offer, and the command carries on without --source.

Never commit the output/ folder

Running the pipeline inside a case tree leaves an untracked output/ there, and pi leaves an untracked .pi/. Publishing tolerates both while they stay untracked. Commit either one and publishing refuses, because it would ship them to every learner.

When the bundler refuses

Each refusal stops the run before anything is written.

Message	What it means	What to do
the output directory is inside the Fito TUI package and will not be written	Your pi session is running in the Fito package directory	Move to a working directory of your own
The output directory exists and holds content that is not a previous bundle … Fito will not overwrite it.	Something other than a bundle sits in ./output/1-ingest/source_bundle/	Move or remove it
source path is not a directory: <path>	The --source path is wrong, or is a file	Correct the path
no ingestible files found under <path>	Every file was excluded or skipped	Check the path; read the table above
the output directory must not equal or contain the source tree	The source sits inside the bundle’s own destination	Point --source somewhere else

If the Fito package is installed without its ingestion skill, the identify command says so and falls back to fetched documentation:

The `fito-source-ingestion` skill is not available yet — falling back to
fetched documentation (`--docs` / `--fetch`).

What’s next

• Identify knowledge components — run the three identify commands over the bundle.
• Fetch documentation from a URL — supplement a tree whose key files were too large to bundle.