core/docs/shopify-import.md

# Shopify → Lunar product import

Findings from comparing a real Shopify product export CSV against Lunar's schema (`vendor/lunarphp/core`), plus the resulting implementation plan for `MigrateImport\Shopify\ShopifyExportImporter`.

## Idempotency problem

Nothing in Lunar tracks "this record came from external system X, ID Y." Re-running an import with no external-ID tracking would duplicate every product on each run.

**Solution: a dedicated `import_mappings` table**, not columns on Lunar's core tables.

```php
Schema::create($this->prefix.'import_mappings', function (Blueprint $table) {
    $table->id();
    $table->string('source');           // 'shopify'
    $table->string('source_type');      // 'product', 'variant', 'image', 'collection', 'tag'
    $table->string('external_id');      // Shopify handle, or handle+position for variants/images
    $table->morphs('model');            // model_type + model_id -> Product, ProductVariant, Asset, Collection, Tag
    $table->timestamps();

    $table->unique(['source', 'source_type', 'external_id']);
});
```

Every resolver looks up `import_mappings` by `(source, source_type, external_id)` first — update if found, create + insert mapping row if not.

## CSV shape

Shopify's export is flat: one row per (product × variant × extra image), with `Handle` repeated across all rows belonging to the same product. Only the first row per handle carries `Title`/`Body (HTML)`/`Vendor`/etc — later rows are variant-only or pure image rows (no `Option1 Value` at all).

The importer must first group consecutive CSV rows by `Handle` into `{ handle, productFields, variants: [...], images: [...] }` before touching Lunar.

## Field mapping: CSV → Lunar

| CSV column | Lunar equivalent | Notes |
|---|---|---|
| `Title`, `Body (HTML)` | `products.attribute_data` (JSON, translatable) | goes into attribute_data, not a flat column |
| `Vendor` | `products.brand` (plain string column) | direct passthrough |
| `Product Category` | `Collection` (nested set) inside a `CollectionGroup` | Google-taxonomy breadcrumb (e.g. `Toys & Games > Toys > Bath Toys`) — parse into a nested Collection chain, product attached to the deepest node. See "Collections" below. |
| `Type` | `product_type_id` (required FK) | must resolve/create a `ProductType`; needs a default fallback (e.g. "General") |
| `Tags` | `Tag` model (separate, not built into Product) | split CSV string, find/create, attach |
| `Published` / `Status` | `products.status` (plain string, indexed) | direct mapping |
| `Option1/2/3 Name+Value` | `product_options` + `product_option_values` + variant pivot | find/create option (by name), find/create value (by option+value), attach to variant |
| `Variant SKU` | `product_variants.sku` | direct |
| `Variant Grams` | `product_variants` dimensions (`$table->dimensions()`) | needs unit conversion check — macro likely adds `weight_value`/`weight_unit`, not raw grams |
| `Variant Inventory Qty` | `product_variants.stock` | direct |
| `Variant Price` / `Compare At Price` | `prices` table (polymorphic `priceable`, per-currency, per-customer-group) | **not a variant column** — create a `Price` row per variant per currency |
| `Cost per item` | *(nothing)* | Lunar has no cost-price field — stash in `attribute_data` as a custom attribute, in scope for v1 |
| `Variant Barcode` | closest is `gtin`/`ean` on variant | ambiguous, needs a mapping rule |
| `Variant Requires Shipping` | `product_variants.shippable` (boolean) | direct |
| `Variant Taxable` | `tax_class_id` (required FK, not boolean) | must resolve to an actual `TaxClass` row (e.g. "Standard" vs "None") |
| `Image Src` (+ image-only rows) | `Asset` model + Spatie MediaLibrary, ordered via `media_product_variant` pivot (`position`) | see "Images" below — file is local, not a URL to download |
| `SEO Title` / `SEO Description` | *(nothing structured)* | stash in `attribute_data` as custom attributes, in scope for v1 |
| Metafield columns (see below) | *(nothing)* | **all empty across every row in the sample export — skip entirely** |
| `Handle` | `Url` model (`urls` table, polymorphic, per-language slug) | create a `Url` row per product (slug = handle), not a plain column |

## Collections — Lunar supports multiple collections per product

Lunar's `collection_product` table is a plain many-to-many pivot (`collection_id`, `product_id`, `position`) with no uniqueness constraint tying a product to a single collection. A `Product` can belong to any number of `Collection`s simultaneously, and `Collection`s themselves are a nested-set tree (`NodeTrait`, `_lft`/`_rgt`/`parent_id`) scoped inside a `CollectionGroup` (`collection_group_id`).

This matters for the importer in two ways:

- **The breadcrumb import doesn't have to be exclusive.** Attaching a product to `Toys & Games > Toys > Bath Toys > Rubber Duckies` doesn't prevent it from also being attached to a separately-curated collection (e.g. a manual "Best Sellers" collection) — no conflict, since it's a plain pivot.
- **Resolve the breadcrumb into whichever `CollectionGroup` fits the target site's structure** — the known Shopify category tree can be parsed straight into it; no need for a separate import-only group unless you specifically want to isolate/purge imported collections independently later.
- Only the **deepest node** of the breadcrumb needs the product attached directly — Lunar's nested-set ancestry (`getBreadcrumbAttribute()`) already exposes the full parent chain via `$collection->ancestors`, so there's no need to attach the product to every intermediate level.

Resolver logic: split `Product Category` on `>`, trim each segment, walk down creating/reusing `Collection` nodes as a chain (`parent_id` chained) within the fixed import `CollectionGroup`, then attach the product only to the final (leaf) `Collection` via `collection_product`.

## Images — the export includes a local files folder

The Shopify export isn't just the CSV — it also produces a folder of the actual image files, named by UUID (not by the original filename or `Image Src` URL). The `Image Src` column in the CSV is the *original* Shopify CDN URL, which is **not** how the file is named on disk in the export folder.

This changes the AssetResolver from an HTTP-download step to a local-file-lookup step:

- The importer needs a way to map each CSV `Image Src` row to its corresponding UUID-named file in the export folder — check whether the export also provides a manifest/mapping file, or whether the mapping has to be inferred from row order per `Handle` (i.e. Nth image row for a handle = Nth file in that handle's image set, if the export preserves ordering).
- Once resolved, the local file is what gets added to the `Asset` + Spatie MediaLibrary (`addMedia($localPath)`), not a downloaded copy of the CDN URL — no HTTP client / retry / timeout handling needed for images, but a **file-existence check** is needed (missing/renamed file in the export folder should not silently fail the whole product import).
- `import_mappings` for images should still key on something stable from the CSV (e.g. `Image Src` URL, or `handle + position`) rather than the UUID filename, since the UUID is regenerated on every fresh export and isn't a stable external ID across re-imports.

## Metafield columns — confirmed empty, skipped

These are Shopify's auto-generated category-specific metafields (populated once a product has a `Product Category` assigned). In the sample export they were empty for effectively every row (a few had `color-pattern`, `material`, or `target-gender` populated). **Decision: skip all of these for v1**, since none carry data worth preserving in this dataset.

```
Product rating count (product.metafields.reviews.rating_count)
Age group (product.metafields.shopify.age-group)
Barware material (product.metafields.shopify.barware-material)
Board game mechanics (product.metafields.shopify.board-game-mechanics)
Color (product.metafields.shopify.color-pattern)
Construction set theme (product.metafields.shopify.construction-set-theme)
Decoration material (product.metafields.shopify.decoration-material)
Dexterity skills (product.metafields.shopify.dexterity-skills)
Dice shape (product.metafields.shopify.dice-shape)
Difficulty level (product.metafields.shopify.difficulty-level)
Doll/Playset features (product.metafields.shopify.doll-playset-features)
Finish (product.metafields.shopify.finish)
Gameplay skills (product.metafields.shopify.gameplay-skills)
Hardware material (product.metafields.shopify.hardware-material)
Material (product.metafields.shopify.material)
Pet trainer type (product.metafields.shopify.pet-trainer-type)
Plant class (product.metafields.shopify.plant-class)
Plant name (product.metafields.shopify.plant-name)
Puzzle theme (product.metafields.shopify.puzzle-theme)
Puzzle type (product.metafields.shopify.puzzle-type)
Recommended age group (product.metafields.shopify.recommended-age-group)
Shape (product.metafields.shopify.shape)
Stationery binding type (product.metafields.shopify.stationery-binding-type)
Suitable space (product.metafields.shopify.suitable-space)
Target gender (product.metafields.shopify.target-gender)
Theme (product.metafields.shopify.theme)
Toy/Game material (product.metafields.shopify.toy-game-material)
Usage type (product.metafields.shopify.usage-type)
Vase shape (product.metafields.shopify.vase-shape)
```

## Resolvers, in dependency order

Each resolver is idempotent: find-by-natural-key-or-create, checking `import_mappings` where applicable.

1. **TaxClassResolver** — maps `Variant Taxable` (bool) to a fixed `TaxClass` ("Standard" vs "None"), not a per-row lookup.
2. **ProductTypeResolver** — maps CSV `Type` to `ProductType`; falls back to a default (e.g. "General") since `product_type_id` is required.
3. **BrandResolver** — `Vendor` → `products.brand`; likely a passthrough, not a real resolver, since it's a plain string column.
4. **TagResolver** — splits `Tags`, finds/creates `Tag` rows, attaches to product.
5. **CollectionResolver** — parses `Product Category` breadcrumb into a nested `Collection` chain within the target `CollectionGroup` (find/create per level), attaches product only to the deepest (leaf) Collection. Does not touch or conflict with any other collections the product may already belong to — a product can be in multiple collections at once.
6. **ProductOptionResolver** — for each `Option1/2/3 Name`, find/create `ProductOption` + `ProductOptionValue`, reused across products.
7. **AssetResolver** — resolves each CSV image row to its local UUID-named file in the export's images folder (see "Images" above), creates `Asset` + Spatie media from that local file, attaches to product and/or specific variants with `position` ordering.
8. **PriceResolver** — creates `Price` row(s) per variant per currency from `Variant Price` / `Variant Compare At Price`.

## Orchestration order per product group

```
1. Resolve/create ProductType, TaxClass, Brand
2. Create or update Product (import_mappings: source_type='product', external_id=handle)
   - write Title/Body, Cost per item, SEO title/description into attribute_data
3. Create/attach Url (slug = handle)
4. Resolve/attach Tags
5. Resolve/attach Collection chain (parsed from Product Category breadcrumb, within the target CollectionGroup; attach product to leaf Collection only)
6. For each Option1/2/3 Name -> resolve ProductOption
7. For each variant row:
     - resolve ProductOptionValues for that variant's option combination
     - create or update ProductVariant (import_mappings: source_type='variant', external_id=handle+variant-index)
     - attach ProductOptionValues to variant
     - create/update Price row(s)
8. For each image row:
     - resolve the CSV row to its local UUID-named file in the export's images folder; skip with a logged warning if the file is missing
     - resolve/create Asset from the local file (import_mappings: source_type='image', external_id=image URL or handle+position — not the UUID filename, which is regenerated per export)
     - attach to Product, and to specific ProductVariant if the image is variant-scoped
```

## Explicitly out of scope for v1

- Metafield columns (confirmed empty — see above).