She did the work and still can't see the mistake. Quiz is
Thursday at 8. This is the moment I care about building for.
02 — Listening first
Ask before answering.
The agent doesn't paste the solution. It asks which part
feels shaky, keeps what it already knows — this week's topic,
the quiz — and breaks the fix into three small steps.
03 — Reading the page
Then it reads the actual work.
OCR is one capability here, not the point: a phone photo of
handwriting becomes text the agent can point at — this line, this
step.
04 — The next step
“Oh. The inside has a slope.”
The win isn't a perfect answer from a machine. It's a
student who can do #4 on her own — and then goes to sleep.
Illustrative scene — not a deployed product or measured result.
Calc · wk 6 · chain rule
#3 f(x) = (3 − x)²
f′(x) = 2(3 − x) ??
key: −2(3 − x) ← why negative
= 6 − 2x ?
inside: u = 3 − x → u′ = −1
f′ = 2(3 − x)·(−1) = −2(3 − x) ✓
L1 · 0.97L2 · 0.91L3 · 0.88
Quiz Thu 8:00 😬
#4 alone then sleep
Name the inside: u = 3 − x
Its slope: u′ = ?you try
Multiply: 2u · u′
11:47 pmi keep getting the sign wrong on #3 and i don't get why.
quiz is thursday
Okay. Before I explain — which part feels shaky: the outside
square, or what's inside it?
the inside i think?
How it's thinking
heard: sign flips on #3
context: wk 6 · chain rule · quiz Thu
suspect: missing inner slope
plan: 3 steps, she does #2
Read from photo · handwriting
L1 f(x) = (3 − x)²
L2 f′(x) = 2(3 − x) ← stops here
L3 key: −2(3 − x)
Line 2 has the outside step. The inside's slope never got
multiplied in.
That's it — and you found the −1 yourself. Same move on #4.
Try it first; I'll only check.
ok. trying #4 before i look 🙏
Tonight
#4 alone · ~10 min
one more like it tomorrow
quiz Thu 8:00
> Fig.01 — the parsing problem
Messy input → LLM-ready context.
01
Messy input
A phone photo, not a clean PDF.
Skewed page, a coffee ring, a margin scribble, a printed
diagram and a running header — the kind of input real students
actually send.
02
Crop & segment
Find the regions that carry meaning.
Layout detection separates body text, the figure and the
handwritten note, then crops each region upright so nothing is
lost to the noise around it.
03
Parse & clean
Turn pixels into structure.
OCR reads the text, broken lines are rejoined, page furniture
is dropped, the equation stays an equation and the figure gets
a caption it can be searched by.
04
Chunk with provenance
Small pieces that remember where they came from.
Text and figure become self-contained chunks, each tagged with
section, page and links to its neighbours — so “see Fig. 3.2”
still points somewhere.
05
Embed
Meaning becomes a position.
Every chunk — text and image description alike — is embedded
into a vector space where related ideas land near each other
and can be found by meaning.
06
Retrieve & answer
The agent answers from the page.
A student's question pulls the nearest chunks, the agent
packs them into context and answers — citing p.42 and Fig. 3.2
instead of guessing.
Visual explanation — not a live tool. Nothing is uploaded or
processed here.
CHAPTER 3 · MOTION 42
3.4 Projectile motion
At the top of its path the ver- tical velocity vy
= 0 . (see Fig 3.2) while vx stays constant…
Fig. 3.2
apex → vy=0 !!
text · 0.98figure · 0.95handwriting · 0.81header · drop
parsed · p.42
## §3.4 Projectile motionAt the top of its path, verticalvelocity v_y = 0 (see Fig. 3.2).[figure 3.2] parabola, apex markedCHAPTER 3 · MOTION 42
✓ Normalized text · regions linked
chunk §3.4#2
At the apex, v_y = 0 …
p.42 · → fig-3.2
chunk fig-3.2
Trajectory, apex where v_y = 0
p.42 · image crop
note · hand
“apex → v_y=0”
p.42 · margin
✓ Schema checked · source preserved
embedding space
[0.12, −0.48, 0.91, …] × 1024
studentWhy is the speed not zero at the
top?
agentOnly the vertical part stops:
vy = 0 at the apex, but vx keeps going.
p.42 §3.4#2Fig. 3.2
> Intro
I'm Michael Le, an AI engineer based in Ho Chi Minh City.
I build systems that meet people where they are — turningquestions, notes and messy documentsinto a useful next step.
Raw text, documents and photos in; clean, structured context out — so
an agent retrieves the right passage or figure and answers from it,
with sources.
Problem space
Real inputs are messy: PDFs, scans, screenshots, notes and photos
with noise, mixed layouts and no structure an LLM can lean on.
Approach
Parse and clean text, crop and describe visual regions, organise
everything into structured chunks, embed them, then retrieve only
what a question needs.
Focus areas
Multimodal parsing, structure an agent understands, semantic
retrieval and grounded answers.
Most of the quality in an agent's answer is decided before the
model ever runs. This is how I approach turning unstructured, multimodal
material into context an LLM can use — seven stages, each with an output
you can inspect.
Traced example One phone photo of a physics
textbook page — p.42, body text, a printed trajectory diagram, a margin
note and a coffee ring — followed through all seven stages to one
student question.
01
Ingest
Take input as it really arrives.
PDFs, scans, screenshots and phone photos arrive in different
formats and quality. Each item is typed, split into pages and
given a source record, so every later chunk can point back to
where it came from.
in
files · text · images
out
typed items + source ids
uses
format detection · page & frame splitting · source
metadata
p.42 at stage 01
beforeIMG_2231.jpg · 3024×4032tilted · shadow · coffee ringno text layer
Text is read (OCR where there is no text layer), then normalised:
hyphenated lines rejoined, running headers and page numbers
dropped, equations and lists kept as structure rather than
flattened prose.
in
raw text · OCR output
out
clean, sectioned text
uses
OCR for photos & scans · de-duplication · layout-aware
cleanup
p.42 at stage 02
beforeCHAPTER 3 · MOTION 42At the top of its path the ver-tical velocity v_y = 0 . (see Fig
after§3.4 Projectile motionAt the top of its path, verticalvelocity v_y = 0 (see Fig. 3.2).
03
Crop & read images
Find the part of the image that matters.
The page is segmented into regions — body text, figure,
handwriting. The diagram is cropped tight to its bounds, then
described and labelled so an agent knows what it shows and which
paragraph cites it.
in
page photo
out
cropped regions + descriptions
uses
region detection · smart cropping · captions & labels
Clean text and image regions merge into chunks with a title, a
place in the hierarchy, a link from the figure to the paragraph
that cites it, and metadata the agent can filter on.
Each chunk, and each figure description, is encoded as an
embedding — a vector of its meaning — and stored with its metadata
in an index built for semantic search.
in
structured chunks
out
vector index
uses
text & image embeddings · metadata filters · re-index on
change
Retrieval is wired in as a tool. The agent decides when to search,
rewrites the question into a better query and chooses filters,
combining what it finds with its own plan and memory.
in
user goal + index
out
retrieval calls
uses
retrieval as a tool · query rewriting · agent memory
p.42 at stage 06
before“why is it 0 at the top??”
aftersearch(“vertical velocity at apex of projectile”, filter: ch 3)
07
Retrieve & respond
Answer with the right context.
Matching chunks are retrieved and re-ranked, packed into the
prompt with their sources, and the model answers from them —
pointing at the page, figure or line it used.