How to refine the gold in your chats: the complete method
AI

⛏️ How to refine the gold in your chats: the complete method

Three phases to turn your AI chat history into a corpus for a personal agent

In the previous article I argued that the gold is in your chats: the corrections accumulated in your history with AI are the raw material for a personal agent’s corpus. After publishing it, I tested the thesis against the literature and amended it: your history does not directly reveal how you think; it records, much more reliably, how you correct.

This article is the second part of that story, and it keeps a promise: the complete extraction method as I executed it, with real numbers, the working prompts from the first two phases, and a reusable formalization of the third, so that anyone with enough history can replicate it.

Your history does not directly reveal how you think; it records, much more reliably, how you correct.

The AI did the extraction

The extraction was done by AI. It did not merely assist the process; it was the fundamental tool. The same AI that accumulated my corrections later searched the history, identified the turns where I corrected it, and distilled the principles. I did not read thousands of messages by hand; I designed the instructions, validated each phase, and retained final judgment.

That division of labor is not a detail: it is the method. The AI brings the ability to read at scale and impose structure; the human brings process design and judgment about the result. If the AI also does the judging, what comes out is a coherent character invented by the model. If the human also does all the reading, the project dies of sheer exhaustion before it is finished.

You do not need to be a programmer. You need a chat history in a tool that lets you search or export it. I used Claude Projects with chat search enabled, a feature available on paid plans and limited to conversations within each individual project. You also need time and one non-negotiable requirement: personally reviewing everything the model distills.

Before you start: data hygiene

The history of anyone doing serious work contains secrets, third-party data, and material that should not circulate. Before processing anything, decide the perimeter: which projects are in, which stay out, and what kind of information gets excluded even when it appears. In my case, I excluded one entire project because of its sensitivity, along with anything involving credentials or data from people outside the process. Make this decision beforehand, with a cool head, not in the middle of the extraction.

Exclusion means keeping that material outside the project or searchable corpus, not relying on an instruction telling the model to ignore it after gaining access.

The extraction workflow can be summarized like this:

History → critical turns → principles → patterns → operational rules.

Phase 1: identify the critical turns

A critical turn is a moment in the history where your judgment was recorded in action. My prompt defined five categories. I am sharing them with minor editorial cleanup that does not change the logic, because the precision of these definitions determines the quality of everything else. My original working prompts were in Spanish; the English blocks below are faithful translations. This is the core of the Phase 1 working prompt:

Your task: search the conversations in this Project using your search
tools. Identify the turns where I said something that meaningfully
reveals my operational reasoning.

Categories of critical turns:

1. EXPLICIT CORRECTIONS: turns where I contradict you, correct you,
   or narrow your approach. Markers: "no," "you're wrong," "stick to,"
   "that's not right."
2. REFRAMINGS: turns where I restate what you said in a way that
   changes its meaning. I do not oppose you directly; I propose another
   way of seeing the same thing.
3. FRAME REJECTIONS: I reject not your answer, but the entire approach.
   Markers: "we've lost focus," "that's not what I want."
4. DEMANDS FOR EVIDENCE: I ask you to justify a claim or show the
   source. Markers: "what are you basing that on,"
   "where did you get that?"
5. STATEMENTS OF PRINCIPLE: I declare a value or a deep preference.
   Markers: "what matters to me is," "this is how I work."

For each turn, return: source conversation, date, category, preceding
context, literal quote (verbatim, without paraphrasing) and resolution.

Rules:
- Do NOT interpret or distill principles yet. Only identify and quote.
- Do NOT discard turns because they seem trivial. I do the filtering.
- If a pattern repeats across conversations, include every instance.
  Repetition is data, not redundancy.

The trick that makes this phase workable is something I learned in practice: topic-based searches fail; searches for your own correction phrases, in your own language and exactly as you write them, hit the mark. Everyone knows their own tells: the phrases you repeat when the model lets you down. Those are the keys to the history.

In my first processed project, this phase produced 48 critical turns from roughly three months of conversations. Then came the first human checkpoint: I validated the list by hand and discarded false positives before moving to the next phase. A literal quote with its source is not bureaucracy; it is what later lets you audit whether the profile is telling the truth.

Phase 2: distill and group

In the pilot, I ran distillation and grouping as separate phases; I later consolidated them into a single prompt that receives the already validated turns. Most of the quality comes from its internal rules. I am also sharing these with minor editorial cleanup:

STEP A - Internal distillation.
For each validated turn, distill the underlying operational principle:
literal trigger quote, context, extracted principle, generalization,
and operational rule for the agent.

Rules:
- ONE PRINCIPLE PER TURN. If a turn contains several, split it into
  separate records.
- DO NOT INVENT. If the principle requires going beyond the turn, mark
  confidence as LOW.
- SEPARATE TOPIC FROM REASONING. What matters is HOW the person
  decides, not what they decided.
- DO NOT COPY LITERAL CONTENT. The quote is verbatim; everything else
  is distillation.

STEP B - Grouping into patterns.
Group the principles into higher-level patterns with: component
principles, evidential strength (`high`, `medium`, `low`), and a
description. Prefer strong consolidation and clean patterns over
forced groupings.

The rule “separate topic from reasoning” is what separates a useful corpus from a technical diary. “He corrected a server command” is not a principle; “technical claims require verification against official documentation before execution” is.

A simplified record moves through the process like this:

Observed topic: corrected a server command → principle: technical claims require official verification before execution → pattern: technical evidence regime → rule: verify every technical claim against official documentation before recommending execution.

One operational lesson I learned the hard way: save each phase’s output to a local file before running the next one. Chats get lost, context windows fill up, and repeating an extraction phase is double work. From that point on, I began adding that warning at the start of each run.

Phase 3: consolidate by recurrence

This process repeats project by project. At the end, cross-project consolidation separates the gold from the sand: An isolated correction may reflect a specific circumstance; a principle that appears repeatedly across different contexts is one of your criteria.

Phase 3 did not run from a saved prompt as Phases 1 and 2 did. I carried it out as a directed session and documented the method in the consolidated file’s methodological notes. The block below formalizes that process as a reusable, anonymized prompt: every rule reflects what I did; it adds no invented steps.

Context: you have N Phase 2 files, one for each processed project.
Each file contains reasoning patterns with: ID, title, component
principles, evidential strength (`high`, `medium`, `low`), and trigger
quotes with their project of origin.

Your task: consolidate all patterns into a single cross-project
document describing the user's operational profile.

Method:
1. Read EVERY pattern from EVERY project, one by one.
   Do not work from summaries or samples.
2. Group by cross-project thematic similarity: patterns from different
   projects with equivalent descriptions represent the same criterion
   appearing in different contexts.
3. Classify each group by recurrence:
   - Present in 4 or more projects: cross-project META-PRINCIPLE.
   - Present in 2 or 3 projects: consistent PRINCIPLE.
   - Present in 1 project: DOMAIN-SPECIFIC PATTERN. It is valid only
     in that context; do NOT generalize it.

Structure of the final document:
- Part I: cross-project meta-principles, each with the projects where it
  appears and one representative quote per project.
- Part II: consistent principles, using the same format.
- Part III: domain-specific patterns grouped by domain.
- Part IV: NEGATIVE EVIDENCE AND PROFILE BOUNDARIES. Record explicitly
  checked behaviors or criteria that are not sufficiently supported by
  the complete Phase 2 pattern set. Treat them as limits of the
  extraction, not as claims about what the person never does.
- Part V: actionable operational rules for the agent's identity layer
  or, depending on the architecture, its system prompt, derived from
  Parts I and II. Use the imperative form, one rule per line.
- Appendix: source-pattern inventory and methodological notes.

Rules:
- DO NOT INVENT. Every group must trace back to source-pattern IDs.
- Evidential strength is inherited from Phase 2. If grouped patterns
  carry different strengths, record the range; do not average them.
- Prefer strong consolidation and clean groups over forced groupings.
  A pattern that does not fit remains on its own.
- DOCUMENT THE LIMITATIONS at the end: missing projects, sample bias,
  and the fact that grouping involves interpretation.
- Treat the document as a living artifact, not a closed contract:
  include a section for future iteration and items that still need
  validation.

These thresholds reflected my eight-project corpus: they are operational cutoffs, not statistical or universal boundaries. With a different number of projects, equivalent thresholds should be defined and documented before consolidation begins.

In that corpus, the 83 extracted patterns were classified into those three levels: patterns present in 4 or more projects became cross-project meta-principles (9 in total); those present in 2 or 3 became consistent principles (6); and those unique to one project remained domain-specific, valid only in their original context.

Negative evidence requires caution: the absence of a behavior from the history does not prove that you never exhibit it. I therefore documented which behaviors or criteria the complete Phase 2 pattern set did not sufficiently support attributing to me. These are limits of the extraction, not claims about what I never do. Defining what the corpus does not support reduces the risk of the model filling the gaps with a coherent character who is not you.

The final product of my extraction was this: 9 meta-principles, 6 consistent principles, domain-specific patterns across 6 domains, and 41 operational rules ready to become the agent’s lightweight identity layer. The strongest of the 83 patterns, with no contradictions detected during the period analyzed, was the evidence regime: never assume, never invent, always cite the source. That the dominant finding of a method built on corrections turned out to be a criterion about how to correct is no coincidence; it is the amended thesis confirming itself.

After extraction: feeding Hermes

Phases 1 through 3 end with the consolidated document. Feeding Hermes was a later, separate stage. It did not mean copying the entire consolidated document into every turn’s context; it meant deciding what should live in the agent’s identity and what should remain in its library.

In my architecture, Hermes is the runtime environment and Nova is the agent operating within it.

1. The distilled rules live in the identity

I integrated the consolidated operational rules into SOUL.md. In my Hermes setup, Nova loads that file as its identity on every turn. This is the lightweight, always-present layer: it contains the criteria that should guide the response, not all the evidence from which they were derived.

2. The full corpus lives in a queryable skill

I deployed the full corpus as the crp/extraction-corpus skill, with stable IDs so Nova could query it on demand and answer with traceability. In May 2026, I validated the flow end to end: Nova retrieved meta-principle M1 and returned the verbatim quotes supporting it together with their source projects.

3. The foundational documents live in the reference library

My first integration mixed those layers. The complete foundational documents were injected on every turn, pushing the recurring context to roughly 72,000 characters. In June 2026, I moved them to ~/.hermes/reference/ as on-demand reference material and kept only minimal versions in active memory. Per-turn context fell to about 13,000 characters.

The corpus is not injected; it is queried.

The agent carries only the lightweight, always-present distilled rules and searches the full corpus when it needs evidence. Feeding Hermes was not about copying a document into it; it was about designing the boundary between identity and library.

The consolidated document also retains value independent of the agent: it is an explicit record of your operational criteria, something most people have never put in writing.

The limitations

There are three, and it pays to know them before you start. The first is retrieval bias: searching for your own phrases finds memorable corrections and misses quiet ones, so the profile overrepresents your most visible rejections. The second is that the distilling model imposes coherence: humans are inconsistent, and the distiller will tend to clean up your contradictions. That is why personally reviewing every record is not optional and why the prompts carry “DO NOT INVENT” in capital letters.

The third is scope: extraction produces an auditable operational profile; by itself, it does not show that an agent retrieves it correctly or that the profile predicts your reasoning beyond the chats. Retrieval must be validated separately, as I did with Nova during the integration stage. I have not validated the extrapolation beyond chat history.

The studies I reviewed examine reasoning personalization from user history, dynamic profiles derived from prior interactions, and contextual preferences revealed through multi-turn feedback, but they do not validate this specific workflow on a real personal archive. What extraction gives you is an operational profile of your correction criteria. For a personal agent, that is exactly what you need. For knowing yourself, it is a good start, not an X-ray.

Refine your own gold and share it

The extraction method is complete in these lines and requires no specialized infrastructure, but it does require searchable or exportable access to your conversation history. The agent implementation can vary; the architectural principle should not: distilled rules in the identity, complete evidence in a queryable library. If you apply it, document your numbers and share what you find: which patterns appeared, which ones surprised you, and where the method failed. That act of sharing is not a courtesy; it is the part of the system that keeps common knowledge alive. In the next article in this series, we will look at a recent study and see why that matters more than it seems.

By: Cesar Rosa Polanco - Written from firsthand experience, with artificial intelligence used as an editorial support tool.

First time here?

Explore the key topics and articles on this blog.

Start Here →