diff --git a/docs/shopify-import.md b/docs/shopify-import.md index 8534085..5a477db 100644 --- a/docs/shopify-import.md +++ b/docs/shopify-import.md @@ -2,6 +2,9 @@ Findings from comparing a real Shopify product export CSV against Lunar's schema (`vendor/lunarphp/core`), plus the resulting implementation plan for `MigrateImport\Shopify\ShopifyExportImporter`. +Need to discard everything and re-import from scratch (e.g. after a schema/indexer change that +only applies to newly-created rows)? See `docs/shopify-reimport.md`. + ## Idempotency problem Nothing in Lunar tracks "this record came from external system X, ID Y." Re-running an import with no external-ID tracking would duplicate every product on each run. diff --git a/docs/shopify-reimport.md b/docs/shopify-reimport.md new file mode 100644 index 0000000..a5de9b3 --- /dev/null +++ b/docs/shopify-reimport.md @@ -0,0 +1,149 @@ +# Wiping products before a clean Shopify re-import + +A runbook for discarding every imported product (and everything that hangs off one — +variants, prices, media, reviews, options/values, the Meilisearch documents) and re-running +`ShopifyExportImporter` from scratch. Useful after a schema/indexer change that only applies to +newly-created rows (see "Why a wipe, not an update" below), or when the export CSV itself changed +enough that stale products need to go, not just be updated in place. + +Every command below is a `tinker --execute=` one-liner run inside the app container — adjust the +exec prefix (`./bin/dc-core.sh exec app ...`, `docker compose exec app ...`, etc.) for your setup. + +--- + +## Why a wipe, not an update + +`ShopifyExportImporter`'s resolvers are mostly `firstOrCreate` — re-running the importer against +an *existing* database updates matched rows but leaves already-created ones exactly as they were. +That's the right behavior for routine re-imports (an updated price, a new variant), but it means a +change to what gets set **at creation time only** — e.g. `ProductOptionResolver` now also setting +`label`, not just `name`, on a `ProductOption` — never reaches a `ProductOption` row that already +exists. A wipe forces every row to go through creation again, picking up such fixes. + +--- + +## 1. Delete every product + +Cascades to `ProductVariant`, prices, and Spatie media rows — verified live (see +`shopify-import.md`'s own history/commit log for context). Also removes each product's Meilisearch +document automatically, via Scout's own delete hook fired on `forceDelete()` — no separate +`scout:flush` needed. + +```php +\Lunar\Models\Product::withTrashed()->get()->each->forceDelete(); +``` + +**Let this run to completion.** Interrupting it mid-loop (e.g. Ctrl+C on the tinker session) stops +after whichever product it was on, leaving the rest undeleted — safe to just re-run the same +command again afterward, since already-deleted products are simply skipped. + +Verify: + +```php +\Lunar\Models\Product::withTrashed()->count(); // 0 +``` + +### Requires: `product_reviews.product_id` cascades on delete + +`product_reviews` (boboko-core's own table, not Lunar's) originally had no `ON DELETE` clause on +its `product_id` foreign key — deleting a reviewed product threw a constraint violation instead of +the review going with it. Fixed by +`database/migrations/2026_09_03_000001_add_cascade_delete_to_product_reviews_product_id.php`. Make +sure this migration has actually run (`php artisan migrate`) before step 1, or a product with +reviews will fail to delete. + +--- + +## 2. Delete product options and values + +Not touched by step 1 (`ProductOption`/`ProductOptionValue` aren't scoped to one product — they're +shared across the catalog, per `ProductOptionResolver::resolveOption()`'s `shared: true`). Safe to +delete in full once every product (and therefore every variant referencing an option value via the +`product_option_value_product_variant` pivot) is gone — deleting values while variants still +reference them throws the same kind of FK violation step 1 guards against. + +```php +\Lunar\Models\ProductOptionValue::query()->delete(); +\Lunar\Models\ProductOption::query()->delete(); +``` + +Verify: + +```php +\Lunar\Models\ProductOption::count(); // 0 +\Lunar\Models\ProductOptionValue::count(); // 0 +``` + +--- + +## 3. Clear the import mappings + +Without this, the importer's `ImportMapping::resolve(...)` calls still find the (now-deleted) +mappings' rows absent, so this step is really about not leaving stale mapping rows pointing at +nothing — `ImportMapping` rows aren't foreign-keyed to the models they map (`morphTo`, no +constraint), so leaving them wouldn't break the re-import, but a stale mapping for a product that +no longer exists is dead weight. + +```php +\Modules\Core\MigrateImport\Models\ImportMapping::where('source', 'shopify')->delete(); +``` + +Verify: + +```php +\Modules\Core\MigrateImport\Models\ImportMapping::where('source', 'shopify')->count(); // 0 +``` + +--- + +## 4. Re-run the importer + +`boboko:migrate:import` dispatches `RunMigrateImportJob` onto the queue — **not synchronous** — +so a queue worker must actually be running (`php artisan queue:work`, or your dev queue container) +or the job just sits queued. + +```bash +php artisan boboko:migrate:import --source=shopify --type=export --file= +``` + +The `--file` value must be an **absolute path** inside the container (e.g. +`/var/www/html/storage/app/private/imports/shopify/products_export.csv`) when running +non-interactively — a path relative to `storage/app/private/imports` only resolves correctly when +the command can fall back to its interactive prompt, which isn't available in a scripted/non-TTY +run. + +Watch the queue worker's own log output for `FAIL` entries (see `docs/lunar.md` or your compose +setup for how logs are routed to `docker compose logs`) — a clean run shows every +`Laravel\Scout\Jobs\MakeSearchable` / `Spatie\MediaLibrary\Conversions\Jobs\PerformConversionsJob` +line ending `DONE`, never `FAIL`. + +--- + +## 5. Re-sync Meilisearch and reindex + +```bash +php artisan lunar:meilisearch:setup +php artisan lunar:meilisearch:tune-product-search +php artisan lunar:search:index "Lunar\Models\Product" --refresh +``` + +`--refresh` re-syncs filterable/sortable index settings *and* reindexes every document — it does +not reset `typoTolerance`/`prefixSearch` (confirmed live: both survived a `--refresh` run +unchanged), so `tune-product-search` only needs re-running here for completeness/if it hadn't +already been applied, not because `--refresh` would have clobbered it. + +--- + +## Verifying the result + +```php +// Product count should match the CSV's actual unique `Handle` count, not +// whatever the database held before the wipe — those aren't the same number +// if stale/manually-added products existed alongside the CSV-sourced ones. +\Lunar\Models\Product::count(); + +// Spot-check that at least one variant picked up its own image (see +// shopify-import.md's "Images" section) — 0 is only correct if the CSV +// genuinely has no `Variant Image` values populated. +\Lunar\Models\ProductVariant::has('images')->count(); +``` diff --git a/src/MigrateImport/Shopify/Resolvers/ProductOptionResolver.php b/src/MigrateImport/Shopify/Resolvers/ProductOptionResolver.php index e4582f4..3bb9b0e 100644 --- a/src/MigrateImport/Shopify/Resolvers/ProductOptionResolver.php +++ b/src/MigrateImport/Shopify/Resolvers/ProductOptionResolver.php @@ -16,10 +16,16 @@ class ProductOptionResolver // the same option instead of creating a near-duplicate. $handle = Str::slug($name) ?: 'option'; + // 'label' must be set even though nothing here reads it back — a null + // label crashes Lunar's own ProductOptionIndexer::toSearchableArray() + // (foreach (null as ...)) the moment this option gets reindexed, since + // it assumes every ProductOption always has one. Same value as 'name' + // is a reasonable default; Shopify's CSV has no separate "label" concept. return ProductOption::query()->firstOrCreate( ['handle' => $handle], [ 'name' => [DefaultLocale::code() => $name], + 'label' => [DefaultLocale::code() => $name], 'shared' => true, ], );