SQL data profiling: 8 queries I run on a brand-new table before anyone writes a test

Someone points you at a table. It showed up in staging last week, it has a sensible name, and the request is short: "Can we build the weekly report off this?" Nobody has written requirements against it yet. There are no tests. The person who built the feed has moved to another project.

This moment matters more to me than any other, and it isn't the one I wrote about in my 7 SQL checks before UAT sign-off post. That post assumed you already knew what "right" looks like: the mandatory fields, the list of allowed status values, the source report you tie out to. On day one you know none of that. You aren't testing the table yet. You're interviewing it.

This post covers SQL data profiling of a table you've never seen before, and the step most profiling guides stop short of: turning the results into questions for the people who actually know the answers. Each query below returns rows only when something needs a question. I owe that habit to Manuel Diaz, who suggested in the comments on my SQL checks post that every check should be a query that returns rows only when something is wrong. I ran all of them on a small fictional table and pasted the real output below.

My take, up front, so you can argue with it in the comments: don't hold the requirements workshop until you've profiled the table, and bring the profile output into the room. A requirement written against a table nobody has profiled is a guess with a ticket number.

Photo by Lukas on Unsplash (Unsplash License). A new table looks like this before you profile it: plausible, and silent about everything that matters.

Why profile before anyone asks for tests

Two new surveys from this year are relevant here. Neither one is about data profiling directly, but both point at the problems profiling is meant to surface.

dbt Labs published its 2026 State of Analytics Engineering report in April 2026, based on 363 practitioners and leaders. It found that 71% of data professionals are concerned about incorrect data reaching stakeholders. Trust in data as an organizational priority went from 66% to 83% in a year, and 41% named ambiguous data ownership as a governance obstacle. Read those last two numbers together. Everybody wants to trust the data, and almost half can't say for sure who owns it.

In Joe Reis's 2026 State of Data Engineering survey (the post says 1,101 responses in the body and 1,001 in the subtitle, so I treat the exact count loosely), 19% named poor requirements as the biggest bottleneck, behind legacy systems at 25% and lack of leadership direction at 21%. On the data modeling side, 51% cited lack of ownership as a pain point.

Poor requirements and unclear ownership are BA problems, and a day-one profile goes straight at both. It shows you what the data actually does, and every finding needs a name next to it. That second part is where the ownership gap shows up, because you'll learn quickly which questions nobody can answer.

There's a newer reason too. AI agents now profile tables on request. Astronomer's public agents repo includes a profiling-tables skill for coding agents, and its published description covers metadata, row counts, per-type column stats, top-20 value frequencies and sample rows, then a quality assessment with scores out of 10 for completeness, uniqueness and freshness, plus a list of potential issues. That's a reasonable list. But a score out of 10 doesn't tell you who to call. DataKitchen's 2026 open source data profiling roundup (updated 22 October 2026) makes a related point from the tooling side: most open source profilers describe data but don't turn the findings into checks anyone enforces. (DataKitchen sells one of the tools that does, so weigh that.) The description is the cheap part now. The real work is deciding what each finding means for your report, and that takes someone who knows the business.

The fixture: a fictional Northstar Systems order-lines feed

To keep this honest, I built a tiny table in SQLite (version 3.53.1) and planted problems in it. Northstar Systems is a fictional client, and every row is made up.

• ns_order_lines: 30 rows, loaded across five weekdays, 26 to 30 October 2026. Columns: order_id, line_no, customer_id, order_date, ship_date, region, status, currency, qty, line_amount, load_date.
• ns_customers: 10 customers, the parent table.
• ns_source_control: one row per load date with the row count and amount total the source system says it sent.

That third table matters. If the source can't provide control totals, asking for them is your first question for stakeholders, before you write a single query.

Here's the fixture, trimmed down to its shape. The full insert script is short enough to copy, and you should run it yourself.

CREATE TABLE ns_customers (customer_id TEXT PRIMARY KEY, customer_name TEXT);
-- 10 rows: C-0101 .. C-0110

CREATE TABLE ns_order_lines (
  order_id TEXT, line_no INTEGER, customer_id TEXT,
  order_date TEXT, ship_date TEXT, region TEXT, status TEXT,
  currency TEXT, qty INTEGER, line_amount REAL, load_date TEXT);

-- sample rows
INSERT INTO ns_order_lines VALUES
 ('NS-1001',1,'C-0101','2026-10-24','2026-10-25','East','Shipped','USD',2,240.00,'2026-10-26'),
 ('NS-1003',1,'C-0103','2026-10-25',NULL,'Central','Open','USD',1,89.99,'2026-10-26'),
 ('NS-1008',1,'C-0199','2026-10-26','2026-10-27','East','SHIPPED','USD',1,99.00,'2026-10-27'),
 ('NS-1011',1,'C-0110','2026-10-27','2026-10-28','','Shipped','USD',2,198.00,'2026-10-27');
 -- ... 30 rows in total

CREATE TABLE ns_source_control (load_date TEXT, src_rows INTEGER, src_amount REAL);
INSERT INTO ns_source_control VALUES
 ('2026-10-26',7,1537.74),('2026-10-27',7,1517.30),('2026-10-28',6,672.00),
 ('2026-10-29',6,1220.00),('2026-10-30',7,612.40);

The one query I run before the profile isn't a check, so it doesn't follow the rows-only-when-wrong rule: SELECT COUNT(*) FROM ns_order_lines returned 30. I write that number down with the date and time, because every later finding is a fraction of it.

Name the grain before you count anything

ontomato summed it up in a comment on Dev.to: name the grain first, as a sentence. "One row = one order line." Until you've written that sentence and tested it, every count you produce is a count of something you haven't defined.

I write down the grain I think the table has, then try to break it.

-- P1 Grain: does (order_id, line_no) identify one row?
-- Returns a row for every candidate key that appears more than once.
SELECT order_id, line_no,
       COUNT(*)                 AS copies,
       GROUP_CONCAT(load_date)  AS load_dates
FROM ns_order_lines
GROUP BY order_id, line_no
HAVING COUNT(*) > 1
ORDER BY order_id, line_no;

Real output:

order_id  line_no  copies  load_dates
--------  -------  ------  ---------------------
NS-1012   1        2       2026-10-28,2026-10-29
NS-1013   1        2       2026-10-28,2026-10-29
NS-1014   1        2       2026-10-28,2026-10-29

Three keys, each loaded twice: once on the 28th and again on the 29th. That pattern is the interesting part. Random duplicates point at the source. Duplicates that cluster on one pair of load dates point at a rerun that appended instead of replacing. Alex Shev made this point in a comment on Dev.to: reruns have to be idempotent, and boundary records are where they usually aren't.

The profile doesn't tell you which copy is right. It only tells you there's a question.

A related trap comes from panditshivam's post on defining revenue for an SQL agent: distinct values are not distinct orders. This table has 30 rows, 23 distinct order_id values and 27 distinct order lines. Ask three people "how many orders came in last week?" and you can get all three numbers back, each one defensible.

Row counts by load date

A new table's history is short, so I compare each day against the average of the other days and flag anything under half or over 1.5 times that average. I also generate the calendar, so a weekday with zero rows shows up instead of quietly not existing.

-- P2 Row counts by load date: spikes, drops and missing weekdays
WITH RECURSIVE days(d) AS (
  SELECT MIN(load_date) FROM ns_order_lines
  UNION ALL
  SELECT DATE(d, '+1 day') FROM days
  WHERE d < (SELECT MAX(load_date) FROM ns_order_lines)
),
daily AS (
  SELECT d AS load_date,
         (SELECT COUNT(*) FROM ns_order_lines o WHERE o.load_date = d) AS row_count
  FROM days
  WHERE STRFTIME('%w', d) NOT IN ('0','6')
),
scored AS (
  SELECT load_date, row_count,
         ROUND((SUM(row_count) OVER () - row_count) * 1.0
               / (COUNT(*) OVER () - 1), 2) AS avg_other_days
  FROM daily
)
SELECT load_date, row_count, avg_other_days,
       CASE WHEN row_count = 0 THEN 'no rows'
            WHEN row_count < 0.5 * avg_other_days THEN 'drop'
            ELSE 'spike' END AS flag
FROM scored
WHERE row_count = 0
   OR row_count < 0.5 * avg_other_days
   OR row_count > 1.5 * avg_other_days
ORDER BY load_date;

Real output:

load_date   row_count  avg_other_days  flag
----------  ---------  --------------  -----
2026-10-29  9          5.25            spike
2026-10-30  1          7.25            drop

The spike on the 29th is the rerun from P1, which is reassuring: two queries, one cause. The drop on the 30th is new. One row on a Friday, when the other days average seven. That's a partial load, a cut-off time nobody told you about, or a very quiet Friday. Only the source owner knows which.

Why the average of the other days instead of the plain average? With five days of history, one bad day drags the plain average toward itself and hides. Leave-one-out keeps the outlier from grading its own homework.

Null and blank rates, every column

Profile every column, not only the ones you think are mandatory. On day one you don't know which ones are mandatory. That's one of the things you're finding out.

-- P3 Null and blank rates, every column
-- Blank means '' or only spaces. Returns columns with any null or blank.
WITH t AS (SELECT COUNT(*) AS n FROM ns_order_lines),
c AS (
  SELECT 'order_id' AS col, SUM(order_id IS NULL) AS nulls, SUM(TRIM(order_id) = '') AS blanks FROM ns_order_lines
  UNION ALL SELECT 'line_no',     SUM(line_no IS NULL),     0 FROM ns_order_lines
  UNION ALL SELECT 'customer_id', SUM(customer_id IS NULL), SUM(TRIM(customer_id) = '') FROM ns_order_lines
  UNION ALL SELECT 'order_date',  SUM(order_date IS NULL),  SUM(TRIM(order_date) = '') FROM ns_order_lines
  UNION ALL SELECT 'ship_date',   SUM(ship_date IS NULL),   SUM(TRIM(ship_date) = '') FROM ns_order_lines
  UNION ALL SELECT 'region',      SUM(region IS NULL),      SUM(TRIM(region) = '') FROM ns_order_lines
  UNION ALL SELECT 'status',      SUM(status IS NULL),      SUM(TRIM(status) = '') FROM ns_order_lines
  UNION ALL SELECT 'currency',    SUM(currency IS NULL),    SUM(TRIM(currency) = '') FROM ns_order_lines
  UNION ALL SELECT 'qty',         SUM(qty IS NULL),         0 FROM ns_order_lines
  UNION ALL SELECT 'line_amount', SUM(line_amount IS NULL), 0 FROM ns_order_lines
  UNION ALL SELECT 'load_date',   SUM(load_date IS NULL),   SUM(TRIM(load_date) = '') FROM ns_order_lines
)
SELECT col, nulls, blanks,
       ROUND(100.0 * (nulls + blanks) / t.n, 1) AS pct_missing
FROM c, t
WHERE nulls + blanks > 0
ORDER BY pct_missing DESC;

Real output:

col        nulls  blanks  pct_missing
---------  -----  ------  -----------
ship_date  7      0       23.3
region     2      2       13.3

A 23% null rate on ship_date looks alarming until you ask why. My guess was that open orders haven't shipped yet, so I wrote the guess as a query that returns rows only if the guess is wrong:

-- P3b Is the ship_date null explained by status?
SELECT status, COUNT(*) AS null_ship_dates
FROM ns_order_lines
WHERE ship_date IS NULL
GROUP BY status
HAVING status <> 'Open';

Real output:

(no rows)

Every null ship date belongs to an Open line. That's an explained null, and it moves from "problem" to "rule someone should confirm." The region gap is different: two nulls and two blanks, one of them two spaces. A plain IS NULL check would have reported half the problem. Two spaces look exactly like an empty cell in most query tools, which is why the TRIM is there.

Distinct counts: constant columns and accidental keys

-- P4 Distinct counts: constant columns and accidental keys
WITH t AS (SELECT COUNT(*) AS n FROM ns_order_lines),
d AS (
  SELECT 'order_id' AS col, COUNT(DISTINCT order_id) AS distinct_vals FROM ns_order_lines
  UNION ALL SELECT 'customer_id', COUNT(DISTINCT customer_id) FROM ns_order_lines
  UNION ALL SELECT 'region',      COUNT(DISTINCT region)      FROM ns_order_lines
  UNION ALL SELECT 'status',      COUNT(DISTINCT status)      FROM ns_order_lines
  UNION ALL SELECT 'currency',    COUNT(DISTINCT currency)    FROM ns_order_lines
  UNION ALL SELECT 'qty',         COUNT(DISTINCT qty)         FROM ns_order_lines
  UNION ALL SELECT 'line_amount', COUNT(DISTINCT line_amount) FROM ns_order_lines
)
SELECT col, distinct_vals, t.n AS total_rows,
       CASE WHEN distinct_vals = 1 THEN 'constant' ELSE 'unique per row' END AS flag
FROM d, t
WHERE distinct_vals = 1 OR distinct_vals = t.n;

Real output:

col       distinct_vals  total_rows  flag
--------  -------------  ----------  --------
currency  1              30          constant

A constant column is one of my favorite findings, because the question it raises is about the future, not the data. Every line is USD. Is Northstar US-only, or is this the first week of a feed that will carry EUR next quarter? If it's the second, every amount total in the report needs a currency rule, and you want that in the requirements now rather than in a defect later.

The other half of the filter, "unique per row," catches a column that turns out to be a key when nobody expected one. Nothing tripped it here. On a real table, it's how you notice that line_amount is actually a free-text comment someone typed numbers into.

Value distributions: rare values and spelling variants

Without an agreed list of allowed values, you can't run an "allowed values" test. What you can do is find the values that look out of place: anything rare, and anything that collides with another value once you upper-case and trim it.

-- P5 Value distributions: rare values and case/space variants
WITH v AS (
  SELECT 'status' AS col, status AS val FROM ns_order_lines
  UNION ALL SELECT 'region', region FROM ns_order_lines
),
counted AS (
  SELECT col, val, COUNT(*) AS cnt,
         ROUND(100.0 * COUNT(*) / SUM(COUNT(*)) OVER (PARTITION BY col), 1) AS pct,
         COUNT(*) OVER (PARTITION BY col, UPPER(TRIM(COALESCE(val, '')))) AS spellings
  FROM v
  GROUP BY col, val
)
SELECT col, QUOTE(val) AS val, cnt, pct,
       CASE WHEN spellings > 1 THEN 'variant' ELSE 'rare' END AS flag
FROM counted
WHERE pct < 5 OR spellings > 1
ORDER BY col, flag, val;

Real output:

col     val          cnt  pct   flag
------  -----------  ---  ----  -------
region  '  '         1    3.3   variant
region  ''           1    3.3   variant
region  NULL         2    6.7   variant
status  'Cancelled'  1    3.3   rare
status  'RTN'        1    3.3   rare
status  'SHIPPED'    1    3.3   variant
status  'Shipped'    20   66.7  variant

QUOTE() is doing quiet work here. Without it, the two-space region and the empty-string region print as identical blank cells, and you'd swear the query had a bug.

The status rows are four different questions. SHIPPED vs Shipped probably means a second source system or a manual fix. RTN might be "returned," and if it is, does a returned line count as revenue? Cancelled is rare, which is fine, but it carries the table's only negative amount (more on that in a second). Rare doesn't mean wrong. It means you should know what a value is before the report counts it.

Min, max and boundary values

I don't print min and max for every column and squint at them. I write down what "plausible" means for this business and let the query return whatever falls outside it.

-- P6 Min, max and boundary values
SELECT order_id, line_no, order_date, ship_date, qty, line_amount, status,
       CASE
         WHEN order_date < '2020-01-01'   THEN 'order_date before 2020'
         WHEN order_date > load_date      THEN 'order_date after load_date'
         WHEN ship_date  < order_date     THEN 'ship_date before order_date'
         WHEN qty <= 0                    THEN 'qty <= 0'
         WHEN line_amount < 0             THEN 'negative amount'
         WHEN line_amount = 0             THEN 'zero amount'
       END AS reason
FROM ns_order_lines
WHERE order_date < '2020-01-01'
   OR order_date > load_date
   OR ship_date  < order_date
   OR qty <= 0
   OR line_amount <= 0
ORDER BY reason, order_id;

Real output:

order_id  line_no  order_date  ship_date   qty  line_amount  status     reason
--------  -------  ----------  ----------  ---  -----------  ---------  ---------------------------
NS-1009   1        2026-10-27  2026-10-27  1    -45.0        Cancelled  negative amount
NS-1020   1        2027-01-15  2026-10-30  1    49.0         Shipped    order_date after load_date
NS-1005   1        1900-01-01  2026-10-26  2    410.0        Shipped    order_date before 2020
NS-1010   1        2026-10-27  2026-10-26  6    720.33       Shipped    ship_date before order_date
NS-1004   2        2026-10-25  2026-10-26  1    0.0          Shipped    zero amount

1900-01-01 is the classic default date: somewhere upstream, a blank became a date. 2027-01-15 on a line that shipped in October is either a typo or a field that means something other than "order date" (a requested delivery date, maybe). The zero-amount line could be a free sample. The negative on a cancelled line could be a reversal, and if it is, someone has to decide whether the report nets it or excludes it.

One thing the output hides: NS-1020 also has a ship date before its order date. The CASE stops at the first reason that matches. For a profile I'm fine with that, because one reason is enough to start a conversation. If you need every reason, swap the CASE for a UNION ALL of one query per rule.

Orphan rate by load date

-- P7 Orphan rate by load date
SELECT o.load_date,
       COUNT(*)                                    AS lines,
       SUM(c.customer_id IS NULL)                  AS orphan_lines,
       ROUND(100.0 * SUM(c.customer_id IS NULL) / COUNT(*), 1) AS orphan_pct,
       GROUP_CONCAT(CASE WHEN c.customer_id IS NULL THEN o.customer_id END) AS orphan_ids
FROM ns_order_lines o
LEFT JOIN ns_customers c ON c.customer_id = o.customer_id
GROUP BY o.load_date
HAVING SUM(c.customer_id IS NULL) > 0
ORDER BY o.load_date;

Real output:

load_date   lines  orphan_lines  orphan_pct  orphan_ids
----------  -----  ------------  ----------  ----------
2026-10-27  7      1             14.3        C-0199
2026-10-29  9      1             11.1        C-0199

Grouping orphans by load date instead of reporting one total came from a commenter, SkepAgent, on the SQL checks post: if orphans cluster after a particular load, the load order was probably wrong, and that's faster to find than chasing records one by one. Here the pattern says something else. It's the same customer, C-0199, on two different days. That looks less like a timing problem and more like a customer who exists in the order system but not in the customer master. Is it a test account? A new customer whose record hasn't synced yet? Those have different owners.

This is the same query from my reconciliation video, pointed at a single table. In the video, orphans are one of the four reasons two dashboards disagree. On day one, they're a question about which system is the master.

https://www.youtube.com/watch?v=SG-6rtZeizg

Video: "Dashboard Numbers Don't Match? Find the Gap in SQL" by Prachit Parikh. It covers definitions, calendar and time zone, grain and fan-out, and orphans. Grain and orphans are the two that show up in a day-one profile.

Control totals vs source

Everything above describes the table. This one checks it against the outside world.

-- P8 Control totals vs source, per load date
WITH tgt AS (
  SELECT load_date, COUNT(*) AS tgt_rows, ROUND(SUM(line_amount), 2) AS tgt_amount
  FROM ns_order_lines GROUP BY load_date
)
SELECT COALESCE(s.load_date, t.load_date) AS load_date,
       s.src_rows, t.tgt_rows,
       s.src_amount, t.tgt_amount,
       ROUND(COALESCE(t.tgt_amount, 0) - COALESCE(s.src_amount, 0), 2) AS amount_diff
FROM ns_source_control s
FULL OUTER JOIN tgt t ON t.load_date = s.load_date
WHERE s.load_date IS NULL OR t.load_date IS NULL
   OR s.src_rows <> t.tgt_rows
   OR ABS(t.tgt_amount - s.src_amount) > 0.005
ORDER BY 1;

Real output:

load_date   src_rows  tgt_rows  src_amount  tgt_amount  amount_diff
----------  --------  --------  ----------  ----------  -----------
2026-10-27  7         7         1517.3      1517.33     0.03
2026-10-29  6         9         1220.0      1672.0      452.0
2026-10-30  7         1         612.4       65.0        -547.4

Three findings, and the profile has already explained two of them. The 29th is the rerun: 452.00 is exactly the three duplicated lines (55 + 267 + 130). To prove that rather than eyeball it, I reran the comparison keeping only the first load of each key:

load_date   src_rows  tgt_rows  src_amount  tgt_amount  amount_diff
----------  --------  --------  ----------  ----------  -----------
2026-10-27  7         7         1517.3      1517.33     0.03
2026-10-30  7         1         612.4       65.0        -547.4

The 29th drops out. The 30th really is a partial load: the source sent seven rows and only one arrived. And the 27th is three cents off even though the row counts match, which is a rounding question. Does the source round each line or the total? What tolerance will the business accept? That tolerance belongs in a written requirement before anyone argues about it during UAT.

Photo by Agence Olloweb on Unsplash (Unsplash License). Profiling is the close look. The questions table is what you carry out of it.

The profile-to-questions table

This is the artifact I actually send. The queries are the evidence, and the table is the agenda. Each row has a finding, the query that found it, the question in plain words, who I'd ask, and what changes depending on the answer.

Some rows don't have a clear owner, and that's a finding too. When I can't fill in the "Who I'd ask" column, I write "unknown" and make finding the owner the first action. Given that 41% figure from dbt Labs, expect a few.

Once the answers come back, most rows turn into one of three things: a requirement, a test, or a known limitation noted in the report's footnotes. That's the bridge into the checks in the 7 SQL checks post and the UAT plan. Because the profile came first, the tests check rules somebody actually confirmed.

The line I'd put on a sticky note: a profile isn't a verdict on the table. It's the agenda for your first meeting about it.

What the sharpest commenter would say

Someone in the comments will raise each of these, so I'll answer them first.

"Thirty rows proves nothing." True. The fixture shows that the queries work and what their output looks like. On a real table with millions of rows, a 5% rarity threshold flags real oddities rather than single rows, and the leave-one-out average needs more than five days to mean much. I'd want at least two full weeks of loads before trusting P2, and where volume is seasonal I'd change it to compare against the same weekday.

"You're profiling staging, but the report reads from the model." Also true, and deliberate. Day one is about the raw feed, because that's where the questions about the source live. Once the model exists, rerun the same eight queries against it. Any finding that disappears was handled by a rule, and you should be able to name that rule.

"SQLite isn't my warehouse." The logic ports. The syntax needs a few swaps: GROUP_CONCAT becomes STRING_AGG in Postgres and SQL Server or LISTAGG in Snowflake, SUM(x IS NULL) becomes SUM(CASE WHEN x IS NULL THEN 1 ELSE 0 END) outside SQLite and MySQL, and the date functions change. FULL OUTER JOIN needs SQLite 3.39 or later.

"Your date comparisons are string comparisons." In SQLite they are, because the dates are stored as text. That only works because they're ISO YYYY-MM-DD. If a feed sends 10/27/2026, P6 will quietly compare garbage. Profiling the format itself (WHERE order_date NOT GLOB '[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]') is a ninth query I'd add the moment I saw a non-ISO date.

"Money in REAL?" The fixture uses REAL to keep things short, and the 0.03 gap shows why that's risky. A real table should store amounts as integer cents or a fixed decimal. If it doesn't, that's another row in the questions table.

"COUNT(DISTINCT) ignores nulls." It does, and it counts '' and ' ' as two separate values. P4 says region has 5 distinct values: three real regions, two flavors of blank, and no mention of the nulls. That's why P5 exists. The distribution check sees the nulls and groups the blanks that distinct counts get wrong in both directions.

What I didn't test

This profile doesn't check whether the numbers are true. A line can pass all eight queries and still carry the wrong price. It doesn't cover cross-table rules beyond one parent table, late-arriving rows that show up days after their load date, or personal data that shouldn't be in the feed at all. And it says nothing about whether the report anyone wants can be built from this table. That's a requirements question, and the profile only makes it a better-informed one.

When to rerun it

After every new load, until the findings stop changing for a week. After any change on the source side. And once more right before UAT, when the confirmed answers have become tests and the profile turns into a regression check: the queries that should now return no rows.

What I'd do on Monday

1. Pick one table you've been asked to report on and haven't profiled yet. Write its grain as a sentence: "One row = one ___."
2. Run P1 against that grain. If it returns rows, stop and ask before you do anything else, because every later count depends on it.
3. Ask the source owner for control totals per load. If they don't exist, that request is your first question, and it's a fair one.
4. Run the rest of the queries, adjusting the column lists and the "plausible" window in P6 to fit the business.
5. Fill in the questions table. Leave the "Who I'd ask" column blank where you don't know, and treat each blank as a task of its own.
6. Send the table before the requirements workshop, not after. Ask people to come with answers.
7. Don't fix anything yet. No TRIM in the model, no deduplication, no defaults. A cleaning rule nobody agreed to is a requirement you wrote by yourself.

That last point is the second thing you might disagree with. It's tempting to clean as you go. But every silent fix is a business decision made by whoever happened to be writing the SQL, and the profile's job is to put those decisions in front of the people who should be making them.

The validation queries I use after the profile, once rules exist, are in sql/data-validation-checks.sql in my BA Toolkit, and the answers from the questions table usually end up in the data mapping template. For the dashboard side of the same problem, the worked revenue reconciliation example shows what happens when nobody names the grain. More of my work is on my portfolio page.

Questions for you

• What's the first query you run on a table you've never seen? Is it on this list?
• Would you bring raw profile output into a requirements workshop, or does it send stakeholders running?
• Have you ever shipped a report where a "cleaning" step turned out to be a business rule nobody agreed to?
• When an AI agent hands you a profile with scores out of 10, what would it need to add before you'd act on it?

Tell me in the comments which finding from the questions table you'd have missed, or which row you'd add. If you have a profiling query that catches something these eight miss, paste it. I'll credit the good ones in a follow-up.

References

• dbt Labs, "New dbt Labs report finds AI-driven acceleration is outpacing trust and governance" (2026 State of Analytics Engineering), April 2026: https://www.getdbt.com/blog/new-dbt-labs-report-finds-ai-driven-acceleration-is-outpacing-trust-and-governance
• Joe Reis, "The 2026 State of Data Engineering Survey (Interactive)", February 2026: https://joereis.substack.com/p/the-2026-state-of-data-engineering
• Chris Bergh, DataKitchen, "The 2026 Open Source Data Profiling Software Landscape", updated 22 October 2026: https://datakitchen.io/blog/the-2026-open-source-data-profiling-software-landscape/
• Astronomer, astronomer/agents repository (includes the profiling-tables skill): https://github.com/astronomer/agents
• "profiling-tables by astronomer" skill description, mcpservers.org: https://mcpservers.org/agent-skills/astronomer/profiling-tables
• ontomato, comment on naming the grain first: https://dev.to/ontomato/comment/3joab
• panditshivam, "Defining revenue for an SQL agent, one row at a time": https://dev.to/panditshivam/defining-revenue-for-an-sql-agent-one-row-at-a-time-a4h
• Alex Shev, comment on control totals, idempotent reruns and boundary records: https://dev.to/alexshev/comment/3jng3

About me

I'm Prachit Parikh, a business analyst. I turn what stakeholders ask for into requirements a team can test, check the data with SQL, and see changes through UAT. I keep free templates, including the SQL data validation checks and a data mapping template, in my BA Toolkit on GitHub, and I make short animated explainers on YouTube. Besides Hashnode, I write on Substack, Medium, Dev.to, Blogger and WordPress, and I answer questions on Quora. You can also find me on LinkedIn, Bluesky, Kaggle, Credly, Crunchbase and Gravatar. Everything is collected on my website and my profile hub.