Article type: How-to guide (one task: export one Shopify store's whole public catalog to a CSV, one row per variant, and check nothing is missing)
Prerequisites: Python 3.9 or later with
requests(pip install requests). No Shopify API key, app or store login: the script reads the public product feed many Shopify storefronts serve. An Apify account only if you use the hosted version at the end.Tested on: 15 November 2026, against one live Shopify store with 509 published products.
Say you need a Shopify store's catalog in a spreadsheet: every product, every variant with its SKU and options, whether it's listed as available, and its images. Maybe it's your own store and you want a quick audit without installing an app, or a supplier's store you reorder from. Many Shopify storefronts publish their products as JSON at /products.json, so a short Python script can page through it, write one CSV row per variant, and confirm the export has every product the store says it has.
The catalog counts and examples in this post come from two public URLs on the store:
-
GET https://{store}/products.json?limit=250&page=Nreturns up to 250 products per page, each with its variants (SKU, options, price,available) and images (URL, size, position). -
GET https://{store}/meta.jsonreturns store details includingpublished_products_count, the number to check the export against.
The example is a 509-product bedding store. It's here only as a public example; any Shopify store whose robots.txt allows /products.json works the same way.
How do I export a Shopify catalog to CSV?
Save this as shopify_catalog.py and run python shopify_catalog.py example-store.com (replace that with any store domain):
import csv
import sys
import time
from collections import Counter
from urllib.robotparser import RobotFileParser
import requests
STORE = sys.argv[1] if len(sys.argv) > 1 else "example-store.com"
BASE = f"https://{STORE}"
UA = "shopify-catalog-csv-tutorial/1.0 (python-requests)"
DELAY = 1.0 # one request a second, or the store's Crawl-delay if longer
session = requests.Session()
session.headers["User-Agent"] = UA
# 1. robots.txt first: stop if the product feed is disallowed.
robots = RobotFileParser()
robots.parse(session.get(f"{BASE}/robots.txt", timeout=30).text.splitlines())
if not robots.can_fetch(UA, f"{BASE}/products.json"):
sys.exit("robots.txt disallows /products.json for this crawler")
DELAY = max(DELAY, float(robots.crawl_delay(UA) or 0))
def get(path, **params):
time.sleep(DELAY)
r = session.get(f"{BASE}{path}", params=params, timeout=30)
r.raise_for_status()
return r.json()
# 2. The store's own product count, to check the export against.
expected = get("/meta.json").get("published_products_count")
# 3. Page through /products.json, 250 at a time, until a page is EMPTY (not just short).
products, seen, page, short_pages = [], set(), 1, []
while True:
batch = get("/products.json", limit=250, page=page)["products"]
if not batch:
break
new = [p for p in batch if p["id"] not in seen]
if not new:
break # a page that only repeats earlier products: stop
if len(batch) < 250:
short_pages.append((page, len(batch)))
seen.update(p["id"] for p in new)
products += new
page += 1
print(f"{STORE}: meta.json says {expected} products; read {len(products)} from {page - 1} pages "
f"(page {page} was empty); short pages: {short_pages}")
# 4. One CSV row per variant, with the product's fields repeated.
rows = []
for p in products:
first_image = p["images"][0]["src"] if p["images"] else ""
tags = p["tags"] if isinstance(p["tags"], list) else [t.strip() for t in p["tags"].split(",") if t.strip()]
for v in p["variants"]:
rows.append({
"product_id": p["id"], "handle": p["handle"], "title": p["title"],
"vendor": p["vendor"], "product_type": p["product_type"], "tags": "|".join(tags),
"variant_id": v["id"], "variant_title": v["title"],
"option1": v["option1"], "option2": v["option2"], "option3": v["option3"],
"sku": (v["sku"] or "").strip(), "price": v["price"], "compare_at_price": v["compare_at_price"] or "",
"available": v["available"],
"variant_image": (v.get("featured_image") or {}).get("src", ""),
"product_image": first_image, "image_count": len(p["images"]),
"url": f"{BASE}/products/{p['handle']}?variant={v['id']}",
"updated_at": p["updated_at"],
})
with open(f"catalog_{STORE}.csv", "w", newline="") as f:
w = csv.DictWriter(f, fieldnames=list(rows[0]))
w.writeheader()
w.writerows(rows)
print(f"wrote {len(rows)} variant rows to catalog_{STORE}.csv")
# 5. Gaps you can see in the public feed. Products priced 0 on every variant are often placeholders,
# samples or free add-ons, so they're counted separately and left out of the checks.
priced = [p for p in products if any(float(v["price"]) > 0 for v in p["variants"])]
ids = {p["id"] for p in priced}
prow = [r for r in rows if r["product_id"] in ids]
print(f"{len(products) - len(priced)} products are priced 0 on every variant; checking the other {len(priced)} "
f"({len(prow)} variants)")
sku_count = Counter(r["sku"] for r in prow if r["sku"])
checks = {
"products with no image": sum(not p["images"] for p in priced),
"products with no product type": sum(not p["product_type"].strip() for p in priced),
"products with no tags": sum(not p["tags"] for p in priced),
"products with every variant unavailable": sum(not any(v["available"] for v in p["variants"]) for p in priced),
"variants with no SKU": sum(not r["sku"] for r in prow),
"SKUs on more than one variant": sum(1 for n in sku_count.values() if n > 1),
"duplicate handles": len(priced) - len({p["handle"] for p in priced}),
"titles used by more than one product": sum(1 for n in Counter(p["title"].strip().lower() for p in priced).values() if n > 1),
}
for name, n in checks.items():
print(f" {name:<42}{n:>6}")
Output on 15 November 2026 from the example store (it took about 9 seconds; the domain is a placeholder):
example-store.com: meta.json says 509 products; read 509 from 3 pages (page 4 was empty); short pages: [(3, 9)]
wrote 5960 variant rows to catalog_example-store.com.csv
202 products are priced 0 on every variant; checking the other 307 (5744 variants)
products with no image 8
products with no product type 0
products with no tags 0
products with every variant unavailable 15
variants with no SKU 18
SKUs on more than one variant 309
duplicate handles 0
titles used by more than one product 2
Some rows from catalog_example-store.com.csv (most columns left out; product names and image counts replaced):
handle,variant_title,sku,available,image_count
cotton-sheet-set,White / Twin,SHT-TW,True,8
cotton-sheet-set,White / Full,SHT-FL,True,8
cotton-sheet-set,White / Queen,SHT-QN,True,8
Why it's built this way:
-
robots.txtfirst. The script stops if the store'srobots.txtdisallows/products.jsonfor it, and uses the store'sCrawl-delayif that's longer than one second. -
One request a second. 509 products took six requests (
robots.txt,meta.jsonand four feed pages). A store with 10,000 products needs 40 feed pages, so at least 40 seconds at this pace, so there's no reason to go faster. -
One row per variant. Variants are what you buy, stock and count, so each gets its own row, with the product's title, type, tags and first image repeated. Filter on
handleto see a product's variants together. -
Check the count. The script prints the store's own
published_products_countnext to the number it read. If they differ, something stopped the export early (or products were published or removed while it ran).
What should I check before I trust the export?
-
limittops out at 250, quietly. A request withlimit=300came back HTTP 200 with 250 products, not an error. If your loop assumes it got what it asked for, it skips products on every page. -
since_iddidn't work here. On this storefront (the only one we tested),/products.json?since_id=<the largest ID on page 1>returned page 1 again, and products aren't in ID order. Page withpage=N. The script also stops if a page only repeats products it already has, so a store that ignores your paging can't loop forever. - Stop on an empty page, not a short one. Here the only short page was the last (page 3, 9 products), and page 4 was empty. The Actor README and our earlier tests say some stores return short pages in the middle of the catalog, so a loop that stops at the first page under 250 can end early and look finished.
- Not every product is for sale. 202 of the 509 products were priced 0 on every variant. 86 of those have the product type "Content" and 93 have no image, so they look like products used to build pages, samples or add-ons rather than things to buy. Counting them would make the audit misleading, so the script reports them separately and checks the other 307 products (5,744 variants). Look at what they are on your own store before deciding.
-
Alt text, barcodes and stock counts aren't in this feed. None of the 16,218 images in this store's
/products.jsonhad analtfield, and none of the 5,960 variants had abarcodeor a stock count. The one product whose own JSON (/products/{handle}.json) we read had bothaltandbarcodethere, at one request per product, but no stock count either. So in the feed,available(true or false) is the only stock signal. -
Tags change shape. In
/products.json,tagswas a list; in the product's own JSON it was a comma-separated string. The script accepts both. - A shared SKU isn't always a mistake. 309 SKUs appeared on more than one variant among the products checked: 258 across different products and 51 within one product. For example, the same blanket SKU appeared on variants of two different blanket products, which can happen when one item is listed in two places. Treat the list as questions to ask, not errors.
-
Prices are strings. Variant prices come back as text, not numbers, so convert with
float()before comparing or summing.
None of the counts in the output says the store did anything wrong. They're the places to look: a product with no image, a variant with no SKU, or a product whose variants are all unavailable might be deliberate, seasonal or simply sold out.
To audit another store, pass its domain. To add alt text and barcodes, request /products/{handle}.json for each product with the same one-second delay, and read images[].alt and variants[].barcode.
The hosted version (Disclosure: we built this)
We built an Apify Actor, Shopify Scraper: Catalog, GTIN Barcode & AI Shopping Audit, that reads the same public storefront JSON for a list of store domains. It pages /products.json until a page is empty, stops if a page only repeats products, and above Shopify's 25,000-product feed limit (from the Actor README and code, not re-tested for this post) reads the rest from the store's public sitemap, collection feeds and product pages. Each product row lists every image and every variant (SKU, price, compare-at price, options, availability), there's a Variants view with one row per variant, and you download it as CSV, Excel or JSON. An audit row per store says whether the whole catalog was read (catalogueComplete) and the store's known product total, plus a 0 to 100 readiness score for missing barcodes, brands, product types, descriptions, alt text, SKUs, images and duplicate titles. It respects robots.txt and Crawl-delay, sends one request a second per store, and backs off when a store answers HTTP 429.
What it doesn't do: barcodes, alt text, weights and stock counts come only from each product's own JSON, so by default they're filled for a sample of 20 products; turn on readAllProductDetails for all of them (one extra request per product, about one a second). It never estimates a stock count the store doesn't publish. It reads 250 products per store unless you set maxProductsPerStore to 0 for the whole catalog, and the default run timeout is 300 seconds, so raise it for very large stores. It doesn't log in, use the Admin API or change anything in a store, and it can't read stores whose product feed is closed (password pages, headless storefronts or bot protection); those, and sites that aren't Shopify, come back as a free status row saying why. It's pay per event; prices on the Actor's Pricing tab. For example (not run for this post):
{
"stores": ["example-store.com"],
"maxProductsPerStore": 0,
"includeCatalogueRows": true,
"includeAudit": true
}
Data: the public Shopify storefront feed of a 509-product bedding store, read 15 November 2026. This post and the Actor are not affiliated with or endorsed by Shopify.
Your Evenings
Top comments (0)