The concept validity studyData
The data
Per-language verdicts
The majority verdict for each language across the five concepts. Every cell resolved to go. The Pages column is the number of completed analyses behind each row; rows marked * (French and Chinese) reached four pages rather than five, below the corpus floor.
| Language | Pages | Boundary | Integ. | Depth | Role | Dim. |
|---|
| French | 4 * | go | go | go | go | go |
| German | 6 | go | go | go | go | go |
| Spanish | 5 | go | go | go | go | go |
| Italian | 6 | go | go | go | go | go |
| Dutch | 5 | go | go | go | go | go |
| Japanese | 5 | go | go | go | go | go |
| Korean | 5 | go | go | go | go | go |
| Chinese | 4 * | go | go | go | go | go |
* Below the pre-registered five-page floor due to crawler blocking; verdicts shown for completeness.
Every page
All 40 native pages, with the retrieval role the tool detected, its coverage score, and the concepts that fell short of a clean go. Where the result reads “all go,” every one of the five concepts was a clean go; “loc” marks needs-localization.
| Lang | Source | Type | Role | Cov. | Concepts below go |
|---|
| FR | editions-tissot.fr | Explainer | explain | 25% | all go |
| FR | ptitchef.com | Guide | guide | 55% | all go |
| FR | selectra.info | Compare | compare | 58% | all go |
| FR | vidal.fr | Explainer | explain | 39% | integration loc |
| DE | easycredit.de | Explainer | explain | 68% | all go |
| DE | heise.de | Explainer | explain | 42% | all go |
| DE | netzwelt.de | Guide | guide | 7% | integration, depth, dimensions loc |
| DE | obi.de | Guide | guide | 57% | all go |
| DE | techbook.de | Compare | compare | 75% | all go |
| DE | verbraucherzentrale.de | Compare | guide | 63% | all go |
| ES | adslzone.net | Guide | guide | 47% | all go |
| ES | consumidorglobal.com | Guide | guide | 66% | all go |
| ES | ocu.org | Compare | compare | 63% | all go |
| ES | pccomponentes.com | Explainer | guide | 39% | all go |
| ES | redeszone.net | Guide | guide | 52% | depth loc |
| IT | altroconsumo.it | Compare | evaluate | 43% | all go |
| IT | focus.it | Guide | explain | 43% | role loc |
| IT | geopop.it | Explainer | explain | 45% | all go |
| IT | html.it | Guide | guide | 80% | depth loc |
| IT | money.it | Compare | compare | 45% | all go |
| IT | punto-informatico.it | Explainer | explain | 62% | all go |
| NL | gratissoftwaresite.nl | Guide | guide | 40% | all go |
| NL | id.nl | Explainer | explain | 62% | all go |
| NL | thuisarts.nl | Explainer | explain | 72% | role loc |
| NL | unitedconsumers.com | Compare | compare | 5% | all go |
| NL | veb.net | Compare | evaluate | 51% | all go |
| JA | furusato-tax.jp | Explainer | guide | 57% | role loc |
| JA | hyponex.co.jp | Guide | guide | 67% | all go |
| JA | propane-npo.com | Compare | evaluate | 51% | all go |
| JA | smbcnikko.co.jp | Compare | compare | 40% | depth loc, role no-go |
| JA | taisho.co.jp | Explainer | explain | 50% | all go |
| KO | 10000recipe.com | Guide | explain | 25% | depth, dimensions loc |
| KO | banksalad.com | Explainer | explain | 75% | all go |
| KO | ohou.se | Compare | compare | 48% | all go |
| KO | oppadu.com | Guide | explain | 63% | all go |
| KO | tossbank.com | Compare | compare | 67% | dimensions loc |
| ZH | 12sporting.com | Guide | guide | 33% | all go |
| ZH | csdn.net | Explainer | explain | 50% | all go |
| ZH | sspai.com | Guide | guide | 45% | role loc |
| ZH | zol.com.cn | Guide | guide | 28% | all go |
Coverage is the page’s overall completeness score, reported by the pipeline and independent of the validity verdicts. Eleven of the 40 pages carried at least one concept below go; the rest were clean across all five.
On confidence intervals
We did not compute bootstrap confidence intervals. With four to six pages per language, the sample is too small for a meaningful resampled interval, and the verdicts are categorical rather than continuous. We report raw majority counts and let the per-page evidence carry the weight. The verdicts also rest on a single Opus 4.8 judge with no human calibration, a trade-off set out on the methodology page.