Mockup for discussion: interactive version of text article 02_04

Text article 02_04 · The 10 Practices of Agentic Data Science: Partnering with AI Agents

Practice 1: Lay Out Your Project So the Agent Can Find Things

The project repo is laid out so that anyone who opens it can find things without asking, and that includes your agent.

Which file would the agent pick?

You've probably opened a project folder like the one below. You can ask a teammate which data file is the real one, but your agent often won't ask. It picks the file that looks most likely and builds on it. You find out it picked the wrong one later, if you even find out at all.

project/

A folder with no obvious place for things

Select a file, or ask to see which one the agent would pick. There's no teammate to ask, and four names that could be "the" data.

In the setup article, the first thing you asked your agent was to list the files in this project and say what each one is for. It could answer because every file in the course repo has a plain name in an obvious place. Putting every file in an obvious place is the first of this course's ten practices: lay out your project so you and the agent can find things.

I first wrote these conventions down back in 2017, in my online book Data Science Bootstrap Notes, for human teammates. It turned out to be prescient; with an agent on the team, they matter even more.

One project, one repo

The rule behind the layout is simple. Each project gets one repository, and everything about the project lives in it: the notebooks, the data and its description, the list of packages, and the instructions for your agent. When there's one place to look, nobody has to remember where the other places are.

For an agent, having one place to look matters even more. Your agent starts every session knowing nothing about your project, and it learns the project by reading the repo. Anything that lives elsewhere, like a spreadsheet on your laptop or a decision made in a meeting, doesn't exist for the agent.

Lives in the repo

The agent can find these, because they're in the one place it reads.

Lives elsewhere

Select one. If it isn't in the repo, it doesn't exist for the agent.

Select any item to see whether the agent can find it.

A tour of the course repo

Select any file or folder to see what lives there and why it sits where it does. You can click, tap, or use the arrow keys.

Pick a file or folder

Each part of the repo has one job and one obvious place. Start with the repo root, or jump to the data folder.

When your project grows

The course repo is small on purpose. A real project grows, and the fuller layout I use gives each project one of each of the pieces below. Add each piece when you need it, not before. Step through them in order, or jump ahead.

The full layout, with a reason for each piece, is in my Data Science Bootstrap Notes.

Spot the stray file

Agents like to leave throwaway files behind, such as a quick_check.py they wrote to test one idea. In a repo where every file has a place, a stray file like that stands out. You can ask the agent why it's there, or delete it.

Find the file that doesn't belong in this course repo.

Try it: ask where the answer came from

Ask your agent:

In one sentence each, what do the pmmov_copies_per_l and population_served columns in the wastewater data mean? Tell me which file you got each answer from. Don't change anything.

Then check two things:

  1. Did the agent name data/README.md?
  2. Open data/README.md yourself. Do its answers match the file?

If the agent answered from memory instead, ask it to check the README. That's a habit worth keeping all course long: when your agent tells you something about your data, ask where it found it.

Practice check

You asked what pmmov_copies_per_l means, and which file the answer came from. Which reply should you trust most?

Chapter 03

Up next, in chapter 03, we'll give the agent the same request with and without a rules file, and compare what it does.