diff --git a/CONTRIBUTE.md b/CONTRIBUTE.md new file mode 100644 index 0000000..d2dc4a1 --- /dev/null +++ b/CONTRIBUTE.md @@ -0,0 +1,67 @@ +# Contributing to boboko-core + +This is a Composer library, not a runnable app — you can't `php artisan serve` it directly. To develop and verify changes, you need a consumer app wired to a local checkout via a Composer path repository, plus a real database, since a large part of this package (Lunar models, migrations, Filament panel resources) can only be meaningfully verified against a live Lunar install. + +## Local dev setup + +This works against any consumer app that follows the same convention — `boboko-test`, `boboko-starter`, `boboko-3dealer`, etc. — checked out next to this repo: + +``` +RadicalElements/ +├── boboko-core/ (this repo) +└── boboko-test/ (or boboko-starter, boboko-3dealer, ... — consumer app, Docker-based) +``` + +Each of these consumer apps ships a `bin/dc-core.sh` helper that wraps the Docker Compose overlay needed to bind-mount a local `boboko-core` checkout into the app container: + +```bash +./bin/dc-core.sh exec app +``` + +This is shorthand for `docker compose -f docker-compose.dev.yml -f docker-compose.core-dev.yml exec app `. Use `./bin/dc-core.sh` for everything below instead of typing the full compose invocation. + +1. **Path repository.** In the consumer app's `composer.json`, the `repositories` array needs a path entry pointing at `../boboko-core`. If it only exists in a disabled block (e.g. `_repositories`), move it into the live array. +2. **Relaxed version constraint.** The consumer app's `composer.json` should require `"boboko/core": "0.*"` (not a tight `^0.0.1` caret) — otherwise Composer rejects newer `0.0.x` versions resolved from the path repo. +3. **Bind mount.** The consumer app's `docker-compose.core-dev.yml` overlays `../boboko-core` into the container at `/var/www/boboko-core`, matching where the path repo resolves it relative to `/var/www/html`. +4. **Re-resolve after every change.** Composer's path repo does not hot-reload — after editing anything in `boboko-core` (including adding new files, which need autoload discovery), the container needs to re-run `composer update boboko/core`. In `boboko-test`, the entrypoint does this automatically on every dev boot (see `docker/entrypoint.sh`), so `./bin/dc-core.sh up` alone picks up local core changes. If a consumer app's entrypoint doesn't do this yet, run it manually: + + ```bash + ./bin/dc-core.sh exec app composer update boboko/core --with-all-dependencies + ``` + + Skipping this step is the most common cause of "my change isn't showing up." + +## Verifying changes against a real database + +There is no automated test suite for this package — too much of Lunar's behavior (table prefixing, nested sets, translatable attributes, Filament panel filters) only breaks in combination, against real Postgres, in a way that's impractical to fake in isolation. Instead, verify changes directly against a consumer app's live database. The practical workflow used throughout this package's `MigrateImport` feature: + +**One-off checks**, via `php artisan tinker --execute="..."` in the consumer app's container: + +```bash +./bin/dc-core.sh exec app php artisan tinker --execute=" +use Modules\Core\MigrateImport\Shopify\Resolvers\ProductTypeResolver; +\$type = (new ProductTypeResolver())->resolve('Test Type'); +echo \$type->name . PHP_EOL; +" +``` + +**Multi-step scripts** (creating related records, checking round-trips), via a raw PHP script bootstrapped like an Artisan command — this avoids tinker's persistent-session quirks and more closely matches how code actually runs in a queued job: + +```bash +./bin/dc-core.sh exec app php -r " +require 'vendor/autoload.php'; +\$app = require 'bootstrap/app.php'; +\$kernel = \$app->make(Illuminate\Contracts\Console\Kernel::class); +\$kernel->bootstrap(); + +// ... your verification code ... +" +``` + +**Always clean up fixtures** created this way — either via `DB::statement('DELETE FROM ...')` in the same script (respecting foreign key order — Lunar has cascading relations like `lunar_customer_group_product` that block naive deletes), or leave them if they're harmless and the DB is a disposable dev/test instance. + +Non-obvious Lunar bugs/traps hit while building against it are documented in [docs/lunar.md](docs/lunar.md#gotchas), not here — check there before debugging something that looks like a Lunar quirk. + +## Code organization + +Feature work lives under `src//`, with models in a `Models/` subdirectory (see `src/Auth/Models`, `src/Customer/Models`, `src/MigrateImport/Models`). Source-specific importer logic is grouped by source name (e.g. `src/MigrateImport/Shopify/`), with resolver classes (one responsibility each — resolve-or-create a single Lunar entity) grouped further under `Resolvers/`. diff --git a/docs/lunar.md b/docs/lunar.md index 58ce4d8..89e8a9e 100644 --- a/docs/lunar.md +++ b/docs/lunar.md @@ -1191,3 +1191,18 @@ php artisan vendor:publish --tag=lunar.migrations # publish migrations php artisan scout:import "Lunar\Models\Product" # index products php artisan scout:flush "Lunar\Models\Order" # flush order index ``` + +--- + +## Gotchas + +Real bugs/traps hit while building against Lunar in this package — not obvious from reading Lunar's source in isolation. + +- **`Lunar\Base\BaseModel` prefixes table names at construction time.** Mass-assigning a `parent_id` on a model using `kalnoy/nestedset`'s `NodeTrait` (e.g. `Collection`) triggers a mutator that queries the database *before* the table prefix is applied, throwing "relation does not exist". Use `appendToNode()`/`saveAsRoot()` instead of setting `parent_id` directly. See `MigrateImport\Shopify\Resolvers\CollectionResolver` for the pattern. +- **Don't trust a schema read from memory — re-check the actual migration/model file.** `products.brand` was assumed to be a plain string column based on an earlier read; it's actually `brand_id`, a real FK to a `Brand` model. Verify column names against the live `Schema::getColumnListing()` or the actual migration file, not recollection. +- **Lunar's default `Language` may not be `en`.** Don't hardcode `app()->getLocale()` for `TranslatedText`/translatable fields — use `Lunar\Models\Language::getDefault()->code` (wrapped here as `MigrateImport\DefaultLocale::code()`). A mismatch means data saves under the wrong locale key and silently doesn't render in the panel. +- **`attribute_data` is not a free-form array.** Values must be `Lunar\Base\FieldType` instances (`Text`, `TranslatedText`, `Number`, etc.), and the handle must be mapped to the product's `ProductType` via `mappedAttributes()`/`productAttributes()` — otherwise the value is silently dropped or won't render in the panel. +- **New `ProductType`s start with zero mapped attributes.** Creating one via `ProductType::create()` alone means `name`/`description` won't work until you `attach()` the existing system attributes to it. +- **`ProductOption.handle` must be unique and non-null if a product has more than one option.** Lunar's Filament variant-switcher widget does `SelectFilter::make($option->handle)` per option — two options with a `null`/matching handle throws "Filter must have a unique name" as a 500 when opening that product's variant pricing page. Always derive a slug and check uniqueness. +- **`Attribute.position` is per-group, and the panel sorts by it.** Hardcoding `position => 1` for multiple new attributes in the same group makes their order undefined/collide with existing attributes at position 1. Compute `max('position') + 1` per group instead. +- **Currency `decimal_places` isn't always 2.** A seeded/demo currency can have the wrong value (seen: EUR seeded with `decimal_places = 1`), which silently corrupts every price display (`€16.50` renders as `165`). If prices look wrong by a factor of 10, check the currency row before assuming the price-writing code is broken. diff --git a/docs/shopify-import.md b/docs/shopify-import.md new file mode 100644 index 0000000..8534085 --- /dev/null +++ b/docs/shopify-import.md @@ -0,0 +1,150 @@ +# Shopify → Lunar product import + +Findings from comparing a real Shopify product export CSV against Lunar's schema (`vendor/lunarphp/core`), plus the resulting implementation plan for `MigrateImport\Shopify\ShopifyExportImporter`. + +## Idempotency problem + +Nothing in Lunar tracks "this record came from external system X, ID Y." Re-running an import with no external-ID tracking would duplicate every product on each run. + +**Solution: a dedicated `import_mappings` table**, not columns on Lunar's core tables. + +```php +Schema::create($this->prefix.'import_mappings', function (Blueprint $table) { + $table->id(); + $table->string('source'); // 'shopify' + $table->string('source_type'); // 'product', 'variant', 'image', 'collection', 'tag' + $table->string('external_id'); // Shopify handle, or handle+position for variants/images + $table->morphs('model'); // model_type + model_id -> Product, ProductVariant, Asset, Collection, Tag + $table->timestamps(); + + $table->unique(['source', 'source_type', 'external_id']); +}); +``` + +Every resolver looks up `import_mappings` by `(source, source_type, external_id)` first — update if found, create + insert mapping row if not. + +## CSV shape + +Shopify's export is flat: one row per (product × variant × extra image), with `Handle` repeated across all rows belonging to the same product. Only the first row per handle carries `Title`/`Body (HTML)`/`Vendor`/etc — later rows are variant-only or pure image rows (no `Option1 Value` at all). + +The importer must first group consecutive CSV rows by `Handle` into `{ handle, productFields, variants: [...], images: [...] }` before touching Lunar. + +## Field mapping: CSV → Lunar + +| CSV column | Lunar equivalent | Notes | +|---|---|---| +| `Title`, `Body (HTML)` | `products.attribute_data` (JSON, translatable) | goes into attribute_data, not a flat column | +| `Vendor` | `products.brand` (plain string column) | direct passthrough | +| `Product Category` | `Collection` (nested set) inside a `CollectionGroup` | Google-taxonomy breadcrumb (e.g. `Toys & Games > Toys > Bath Toys`) — parse into a nested Collection chain, product attached to the deepest node. See "Collections" below. | +| `Type` | `product_type_id` (required FK) | must resolve/create a `ProductType`; needs a default fallback (e.g. "General") | +| `Tags` | `Tag` model (separate, not built into Product) | split CSV string, find/create, attach | +| `Published` / `Status` | `products.status` (plain string, indexed) | direct mapping | +| `Option1/2/3 Name+Value` | `product_options` + `product_option_values` + variant pivot | find/create option (by name), find/create value (by option+value), attach to variant | +| `Variant SKU` | `product_variants.sku` | direct | +| `Variant Grams` | `product_variants` dimensions (`$table->dimensions()`) | needs unit conversion check — macro likely adds `weight_value`/`weight_unit`, not raw grams | +| `Variant Inventory Qty` | `product_variants.stock` | direct | +| `Variant Price` / `Compare At Price` | `prices` table (polymorphic `priceable`, per-currency, per-customer-group) | **not a variant column** — create a `Price` row per variant per currency | +| `Cost per item` | *(nothing)* | Lunar has no cost-price field — stash in `attribute_data` as a custom attribute, in scope for v1 | +| `Variant Barcode` | closest is `gtin`/`ean` on variant | ambiguous, needs a mapping rule | +| `Variant Requires Shipping` | `product_variants.shippable` (boolean) | direct | +| `Variant Taxable` | `tax_class_id` (required FK, not boolean) | must resolve to an actual `TaxClass` row (e.g. "Standard" vs "None") | +| `Image Src` (+ image-only rows) | `Asset` model + Spatie MediaLibrary, ordered via `media_product_variant` pivot (`position`) | see "Images" below — file is local, not a URL to download | +| `SEO Title` / `SEO Description` | *(nothing structured)* | stash in `attribute_data` as custom attributes, in scope for v1 | +| Metafield columns (see below) | *(nothing)* | **all empty across every row in the sample export — skip entirely** | +| `Handle` | `Url` model (`urls` table, polymorphic, per-language slug) | create a `Url` row per product (slug = handle), not a plain column | + +## Collections — Lunar supports multiple collections per product + +Lunar's `collection_product` table is a plain many-to-many pivot (`collection_id`, `product_id`, `position`) with no uniqueness constraint tying a product to a single collection. A `Product` can belong to any number of `Collection`s simultaneously, and `Collection`s themselves are a nested-set tree (`NodeTrait`, `_lft`/`_rgt`/`parent_id`) scoped inside a `CollectionGroup` (`collection_group_id`). + +This matters for the importer in two ways: + +- **The breadcrumb import doesn't have to be exclusive.** Attaching a product to `Toys & Games > Toys > Bath Toys > Rubber Duckies` doesn't prevent it from also being attached to a separately-curated collection (e.g. a manual "Best Sellers" collection) — no conflict, since it's a plain pivot. +- **Resolve the breadcrumb into whichever `CollectionGroup` fits the target site's structure** — the known Shopify category tree can be parsed straight into it; no need for a separate import-only group unless you specifically want to isolate/purge imported collections independently later. +- Only the **deepest node** of the breadcrumb needs the product attached directly — Lunar's nested-set ancestry (`getBreadcrumbAttribute()`) already exposes the full parent chain via `$collection->ancestors`, so there's no need to attach the product to every intermediate level. + +Resolver logic: split `Product Category` on `>`, trim each segment, walk down creating/reusing `Collection` nodes as a chain (`parent_id` chained) within the fixed import `CollectionGroup`, then attach the product only to the final (leaf) `Collection` via `collection_product`. + +## Images — the export includes a local files folder + +The Shopify export isn't just the CSV — it also produces a folder of the actual image files, named by UUID (not by the original filename or `Image Src` URL). The `Image Src` column in the CSV is the *original* Shopify CDN URL, which is **not** how the file is named on disk in the export folder. + +This changes the AssetResolver from an HTTP-download step to a local-file-lookup step: + +- The importer needs a way to map each CSV `Image Src` row to its corresponding UUID-named file in the export folder — check whether the export also provides a manifest/mapping file, or whether the mapping has to be inferred from row order per `Handle` (i.e. Nth image row for a handle = Nth file in that handle's image set, if the export preserves ordering). +- Once resolved, the local file is what gets added to the `Asset` + Spatie MediaLibrary (`addMedia($localPath)`), not a downloaded copy of the CDN URL — no HTTP client / retry / timeout handling needed for images, but a **file-existence check** is needed (missing/renamed file in the export folder should not silently fail the whole product import). +- `import_mappings` for images should still key on something stable from the CSV (e.g. `Image Src` URL, or `handle + position`) rather than the UUID filename, since the UUID is regenerated on every fresh export and isn't a stable external ID across re-imports. + +## Metafield columns — confirmed empty, skipped + +These are Shopify's auto-generated category-specific metafields (populated once a product has a `Product Category` assigned). In the sample export they were empty for effectively every row (a few had `color-pattern`, `material`, or `target-gender` populated). **Decision: skip all of these for v1**, since none carry data worth preserving in this dataset. + +``` +Product rating count (product.metafields.reviews.rating_count) +Age group (product.metafields.shopify.age-group) +Barware material (product.metafields.shopify.barware-material) +Board game mechanics (product.metafields.shopify.board-game-mechanics) +Color (product.metafields.shopify.color-pattern) +Construction set theme (product.metafields.shopify.construction-set-theme) +Decoration material (product.metafields.shopify.decoration-material) +Dexterity skills (product.metafields.shopify.dexterity-skills) +Dice shape (product.metafields.shopify.dice-shape) +Difficulty level (product.metafields.shopify.difficulty-level) +Doll/Playset features (product.metafields.shopify.doll-playset-features) +Finish (product.metafields.shopify.finish) +Gameplay skills (product.metafields.shopify.gameplay-skills) +Hardware material (product.metafields.shopify.hardware-material) +Material (product.metafields.shopify.material) +Pet trainer type (product.metafields.shopify.pet-trainer-type) +Plant class (product.metafields.shopify.plant-class) +Plant name (product.metafields.shopify.plant-name) +Puzzle theme (product.metafields.shopify.puzzle-theme) +Puzzle type (product.metafields.shopify.puzzle-type) +Recommended age group (product.metafields.shopify.recommended-age-group) +Shape (product.metafields.shopify.shape) +Stationery binding type (product.metafields.shopify.stationery-binding-type) +Suitable space (product.metafields.shopify.suitable-space) +Target gender (product.metafields.shopify.target-gender) +Theme (product.metafields.shopify.theme) +Toy/Game material (product.metafields.shopify.toy-game-material) +Usage type (product.metafields.shopify.usage-type) +Vase shape (product.metafields.shopify.vase-shape) +``` + +## Resolvers, in dependency order + +Each resolver is idempotent: find-by-natural-key-or-create, checking `import_mappings` where applicable. + +1. **TaxClassResolver** — maps `Variant Taxable` (bool) to a fixed `TaxClass` ("Standard" vs "None"), not a per-row lookup. +2. **ProductTypeResolver** — maps CSV `Type` to `ProductType`; falls back to a default (e.g. "General") since `product_type_id` is required. +3. **BrandResolver** — `Vendor` → `products.brand`; likely a passthrough, not a real resolver, since it's a plain string column. +4. **TagResolver** — splits `Tags`, finds/creates `Tag` rows, attaches to product. +5. **CollectionResolver** — parses `Product Category` breadcrumb into a nested `Collection` chain within the target `CollectionGroup` (find/create per level), attaches product only to the deepest (leaf) Collection. Does not touch or conflict with any other collections the product may already belong to — a product can be in multiple collections at once. +6. **ProductOptionResolver** — for each `Option1/2/3 Name`, find/create `ProductOption` + `ProductOptionValue`, reused across products. +7. **AssetResolver** — resolves each CSV image row to its local UUID-named file in the export's images folder (see "Images" above), creates `Asset` + Spatie media from that local file, attaches to product and/or specific variants with `position` ordering. +8. **PriceResolver** — creates `Price` row(s) per variant per currency from `Variant Price` / `Variant Compare At Price`. + +## Orchestration order per product group + +``` +1. Resolve/create ProductType, TaxClass, Brand +2. Create or update Product (import_mappings: source_type='product', external_id=handle) + - write Title/Body, Cost per item, SEO title/description into attribute_data +3. Create/attach Url (slug = handle) +4. Resolve/attach Tags +5. Resolve/attach Collection chain (parsed from Product Category breadcrumb, within the target CollectionGroup; attach product to leaf Collection only) +6. For each Option1/2/3 Name -> resolve ProductOption +7. For each variant row: + - resolve ProductOptionValues for that variant's option combination + - create or update ProductVariant (import_mappings: source_type='variant', external_id=handle+variant-index) + - attach ProductOptionValues to variant + - create/update Price row(s) +8. For each image row: + - resolve the CSV row to its local UUID-named file in the export's images folder; skip with a logged warning if the file is missing + - resolve/create Asset from the local file (import_mappings: source_type='image', external_id=image URL or handle+position — not the UUID filename, which is regenerated per export) + - attach to Product, and to specific ProductVariant if the image is variant-scoped +``` + +## Explicitly out of scope for v1 + +- Metafield columns (confirmed empty — see above).