{"id":28566817,"url":"https://github.com/salsify/hotcat","last_synced_at":"2025-06-10T15:39:19.081Z","repository":{"id":5913008,"uuid":"7132228","full_name":"salsify/hotcat","owner":"salsify","description":"ICEcat -\u003e Salsify","archived":false,"fork":false,"pushed_at":"2013-03-14T14:00:21.000Z","size":335,"stargazers_count":0,"open_issues_count":1,"forks_count":1,"subscribers_count":43,"default_branch":"master","last_synced_at":"2024-04-08T15:40:31.462Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/salsify.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-12-12T15:19:33.000Z","updated_at":"2024-04-08T15:40:31.463Z","dependencies_parsed_at":"2022-07-29T18:39:48.833Z","dependency_job_id":null,"html_url":"https://github.com/salsify/hotcat","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salsify%2Fhotcat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salsify%2Fhotcat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salsify%2Fhotcat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salsify%2Fhotcat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/salsify","download_url":"https://codeload.github.com/salsify/hotcat/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salsify%2Fhotcat/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259104095,"owners_count":22805808,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-10T15:39:10.612Z","updated_at":"2025-06-10T15:39:19.066Z","avatar_url":"https://github.com/salsify.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Hotcat\n\nHotcat is a library for accessing and transforming ICEcat data into Salsify's required data ingest format.\n\n## Installation\n\nAdd this line to your Gemfile (probably for development only):\n\n    gem 'hotcat', :git =\u003e 'git@github.com:socialceramics/hotcat.git'\n\nAnd then execute:\n\n    $ bundle\n\n## Usage\n\nHotcat's primary interface is via Rake tasks which require configuration.\n\n### Configuration\n\nConfiguration is done as follows:\n```\n  Hotcat::Configuration.configure do |config|\n    # REQUIRED\n    config.username = \"ICECAT USERNAME\"\n    config.password = \"ICECAT PASSWORD\"\n    \n    # Max # of products to load (not included related products).\n    # -1 for all.\n    # config.max_products = 100\n\n    # Max # of related products PER PRODUCT to load.\n    # 0  for none (just load the products).\n    # -1 for all.\n    # config.max_related_products = 200\n\n    # REQUIRED\n    # This is the local icecat directory where all data will be stored.\n    config.cache_dir = \"/path/to/your/local/icecat/files\"\n\n    # whether or not to use AWS for images\n    config.use_aws_for_images = true\n    config.aws_bucket_id = \"icecat-images-cache\"\n    config.aws_key_id = ENV['AWS_ACCESS_KEY_ID']\n    config.aws_key = ENV['AWS_SECRET_ACCESS_KEY']\n  end\n```\nI recommend you do this in Rails using initializers. Basically put a hotcat.rb file into config/initializers and you'll be good to go.\n\nNote that using AWS could be advantageous if you're doing multiple imports, since it will cache the ICEcat images into an S3 bucket only once for each product. The public URL will then refer to files in this bucket rather than to ICEcat.\n\n### Rake tasks\n\nHotcat comes with a number of rake tasks that are the primary interface for the system. Make sure that your configuration is set up, especially the local cache directory, for this to work appropriately.\n\n#### Primary Tasks\n\nFirst you must download data files from ICEcat to convert.\n\n```bash\nrake hotcat:build_product_camera_cache\n```\n\nThis will download details for up to the maximum number of products specified in the hotcat configuration in the digital camera category.\n\nThere is one main rake task that _should_ take care of everything in one fell swoop once you have the data downloaded.\n\n```bash\nrake hotcat:generate_salsify_import\n```\n\nThis will produce a single file called `salsify-import.zip` in the _salsify_ subdirectory of the cache directory specified in your hotcat configuration. That zipfile can be fed directly to Salsify using one of the two following commands:\n```bash\nrake salsify:clean_load file=/path/to/cache/dir/salsify-import.zip\nrake salsify:load file=/path/to/cache/dir/salsify-import.zip\n```\nThat the first of these runs a *clean_load* which will reset the database. The *load* command is simply additive. If you just want to add to an existing database without resetting you can run *load* with the categories file instead of *clean_load* and everything should still work fine.\n\nIf the salsify-import.zip file exists when the rake task is run, it will be moved to a time-stamped version of the file such as salsify-import-TIMESTAMP.zip before a fresh salsify-import.zip file is generated.\n\n#### Generating Salsify CSV Import Documents\n\nThis is a simple process.\n\n```bash\nrake hotcat:generate_salsify_csv_import\n```\n\nThis will generate a single CSV file that contains both products and accessories for the import.\n\nAs part of this process Hotcat will produce a file called `salsify-attributes_list.txt` in the `/path/to/cache/salsify/` directory. This is the list of all product attributes seen during the conversion. The reason to keep this around is that you may want to re-run the conversion and produce a CSV with fewer columns. Edit this file, and re-run as follows:\n\n```bash\nrake hotcat:generate_salsify_csv_import attributes=/path/to/cache/salsify/salsify-attributes_list.txt\n```\n\nIf you're going to be iterating in this way, you may not want to bother dealing with the image URLs the first time around:\n\n```bash\nrake hotcat:generate_salsify_csv_import load_images=false\n```\n\n#### Other Tasks\n\nThese tasks are rarely run individually as they are called implicitly as needed by other tasks. There are provided separately primarily for debugging.\n\n* **hotcat:load_suppliers**: ensures the ICEcat supplier list is downloaded locally.\n* **hotcat:load_categories**: ensures the ICEcat category list document is downloaded.\n* **hotcat:convert_categories**: grabs the ICEcat category document and converts it to Salsify's data ingest format.\n* **hotcat:build_product_camera_cache**: downloads details for up to the maximum number of products specified in the hotcat configuration in the camera category. The output will be a file called salsify-CategoryList.xml.gz.\n* **hotcat:convert_products**: converts the locally cached products to salsify product documents. This may download additional product detail documents for related products and then load them.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalsify%2Fhotcat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsalsify%2Fhotcat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalsify%2Fhotcat/lists"}