Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/commercetools-demo/demotools

Tools & utilities for building commercetools demos
https://github.com/commercetools-demo/demotools

commercetools

Last synced: 4 months ago
JSON representation

Tools & utilities for building commercetools demos

Awesome Lists containing this project

README 

          
# demotools



Tools to aid in the process of building demos, scripts, etc. for commercetools.  

The primary focus is on transforming product catalog data and importing into the commercetools platform for standing up demos quickly.



## Components



### commercetools.js



Encapsulation of the commercetools platform API client:



- **apiRoot** - Main API client for commercetools Platform API

- **allow404(p)** - Wrapper for an execute function which returns null instead of throwing an error if a 404 is encountered

  - `p`: Promise to execute, typically an API call that might return 404 Not Found

  - Returns: Result of the promise or null if a 404 error occurred



### importApi.js



Client for the commercetools Import API:



- **importApiRoot** - API client for commercetools Import API



### connectApi.js



Client for the commercetools Connect API:



- **connectRoot** - API client for commercetools Connect API



### getAll.js



Functions to retrieve large datasets from commercetools:



- **getAll(args)** - Issues a query that will potentially return large amounts of data, handling pagination automatically

  - Parameters:

    - `args.endpoint`: (Required) The commercetools API endpoint to query (e.g., apiRoot.products(), apiRoot.categories())

    - `args.queryArgs`: (Optional) Object containing query parameters such as:

      - `where`: String or array of where conditions

      - `expand`: String or array of expansion paths

      - `priceCurrency`, `priceCountry`, `priceCustomerGroup`, `priceChannel`: Price selection parameters

    - `args.callback`: (Optional) Function to process results in batches of up to 500 items

    - `args.max`: (Optional) Maximum number of results to return, defaults to all

    - `args.debug`: (Optional) Boolean to enable debug logging

  - Returns: Array of all results (or none if using callback)

  

  Example:

  ```js

  // Get all products

  const products = await getAll({

    endpoint: apiRoot.products()

  });

  

  // Get products with filtering and process in batches

  await getAll({

    endpoint: apiRoot.productProjections(),

    queryArgs: { where: 'categories(id="123")', expand: 'categories[*]' },

    callback: (batch) => processBatch(batch),

    max: 1000

  });

  ```



### getCount.js



Functions for counting large datasets:



- **getCount(args)** - Retrieves an accurate count of items, even when the count exceeds the platform's standard limit of 10,000

  - Parameters:

    - `args.endpoint`: (Required) The commercetools API endpoint to query (e.g., apiRoot.products())

    - `args.queryArgs`: (Optional) Object containing query parameters such as `where` conditions

  - Returns: Total count of matching items

  

  Example:

  ```js

  // Count all products

  const totalProducts = await getCount({

    endpoint: apiRoot.products()

  });

  

  // Count products matching a condition

  const totalProductsInCategory = await getCount({

    endpoint: apiRoot.products(),

    queryArgs: { where: 'masterData(current(categories(id="123")))' }

  });

  ```



### product.js



Product-specific utilities:



- **actionAllProducts(actionCallback, args)** - Perform update actions for all products

  - Parameters:

    - `actionCallback`: (Required) Function that takes a product and returns an array of update actions

    - `args`: (Optional) Object with options:

      - `debug`: Boolean to enable debug logging

      - Can include any parameters accepted by getAll() such as queryArgs

  - Returns: Object with count of updated products and errors: `{count, errors}`

  

  Example:

  ```js

  // Add a custom attribute to all products

  const result = await actionAllProducts(

    (product) => [{

      action: 'setAttribute',

      variantId: product.masterData.current.masterVariant.id,

      name: 'customFlag',

      value: true

    }],

    { debug: true }

  );

  console.log(`Updated ${result.count} products with ${result.errors} errors`);

  ```



- **actionAllVariants(actionCallback, debug)** - Perform update actions for all product variants (master and regular variants)

  - Parameters:

    - `actionCallback`: (Required) Function that takes a product and variant and returns an array of update actions

    - `debug`: (Optional) Boolean to enable debug logging (default: false)

  - Returns: Object with count of updated products and errors: `{count, errors}`

  

  Example:

  ```js

  // Add an attribute to all variants of all products

  const result = await actionAllVariants(

    (product, variant) => [{

      action: 'setAttribute',

      variantId: variant.id,

      name: 'processed',

      value: true

    }],

    true

  );

  console.log(`Updated ${result.count} products with ${result.errors} errors`);

  ```



### productType.js



Product type management utilities:



- **getProductType(key)** - Retrieve a product type by key, returns null if not found

  - `key`: (Required) The product type key to look up

  - Returns: The product type object or null if not found



- **createProductType(body)** - Create a new product type

  - `body`: (Required) Product type definition object

  - Returns: Result of the create operation



- **deleteProductType(key, version)** - Delete a product type by key and version

  - Parameters:

    - `key`: (Required) The product type key to delete

    - `version`: (Required) The version number of the product type to delete

  - Returns: Result of the delete operation



- **updateProductType(key, body)** - Update an existing product type

  - Parameters:

    - `key`: (Required) The product type key to update

    - `body`: (Required) Product type definition object with version

  - Returns: Updated version number if successful, null otherwise



### type.js



Type management utilities:



- **getType(key)** - Retrieve a type by key, returns null if not found

  - `key`: (Required) The type key to look up

  - Returns: The type object or null if not found



- **createOrUpdateType(type)** - Create a new type or update an existing one

  - `type`: (Required) Type definition object

  - Returns: Result of the create or update operation



- **deleteType(key, version)** - Delete a type by key and version

  - Parameters:

    - `key`: (Required) The type key to delete

    - `version`: (Required) The version number of the type to delete

  - Returns: Result of the delete operation



### transform.js



Data mapping and transformation utilities:



- **mapFields(mapperList, input, output, initDebug)** - Maps fields from a source object to a destination object using mapper configurations

  - Parameters:

    - `mapperList`: (Required) Array of mapper objects defining the transformation rules

    - `input`: (Required) Source object containing the data to transform

    - `output`: (Required) Destination object to populate with transformed data

    - `initDebug`: (Optional) Boolean to enable debug logging

  - Returns: No return value; modifies the output object directly

  

- **toSlug(s)** - Creates a commercetools-compatible URL slug from a string

  - `s`: (Required) String to convert to a slug

  - Returns: Slugified string



- **addVariantToProduct(product, variant)** - Adds a variant to a product

  - Parameters:

    - `product`: (Required) Product object to add the variant to

    - `variant`: (Required) Variant object to add to the product

  - Returns: No return value; modifies the product object directly



A Mapper has (up to) five fields:



* **src**: The name of the field in the source object

* **dest**: The name of the field in the destination object

* **convert**: (optional) - a conversion operation to perform. One of:

    slug, category, price, number, boolean, image, list, newline-list, array-list, text, flatten, flatten-list

* **type**: (optional) - the data type of the destination field. One of:

  attr, array

* **locale**: (optional) - for localized fields, specifies the locale (e.g., 'en-US')



Examples:

```js

{

  src: 'ProductID',

  dest: 'key',

}

```

The field *ProductID* in the source object will be mapped to the field *key* in the destination object -- no transformations performed (straight copy)



#### Price

Convert a Price:

```js

{

  src: 'SalePrice',

  dest: 'prices',

  convert: 'price'

}

```



#### Localized Fields:



```js

{

  src: 'ProductName',

  dest: 'name',

  locale: 'en-US'

}

```

In this case, ProductName in the source will map to a localized field *name* in the destination, in a structure used by commercetools localized fields.



#### Attributes

```js

{

  src: 'Department',

  dest: 'department',

  type: 'attr'

}

```

The source field Department will be copied to an attribute called *department* in an *attributes* array of the destination object



#### Category (reference by key)

Use both a 'convert' (to convert to a reference to a category), and a 'type' (array)

```js

{

  src: 'CategoryID',

  dest: 'categories',

  convert: 'category',

  type: 'array'

}

```



#### Images

Convert to an (external) image format

```js

{

  src: 'MainImageURL',

  dest: 'images',

  convert: 'image',

  type: 'array'

}

```



### files.js



File I/O utilities:



- **readCsv(filename, delimiter, quote, verbose)** - Read CSV file and return as an array of objects

  - Parameters:

    - `filename`: (Required) Path to the CSV file

    - `delimiter`: (Optional) Column delimiter character (default: ',')

    - `quote`: (Optional) Quote character (default: '"')

    - `verbose`: (Optional) Boolean flag to enable verbose logging (default: false)

  - Returns: Array of objects with properties named by headers from the first row



- **writeCsv(filename, data, options)** - Write array of objects to a CSV file

  - Parameters:

    - `filename`: (Required) Path to write the CSV file

    - `data`: (Required) Array of objects to write

    - `options`: (Optional) Object with options:

      - `headers`: Array of column headers (if not to be auto-generated from first object)

      - `delimiter`: Column delimiter character (default: ',')

  - Returns: No return value



- **readJSON(filename)** - Read JSON file and return parsed object

  - `filename`: (Required) Path to the JSON file

  - Returns: Parsed JavaScript object



- **writeJSON(filename, data)** - Write object to a JSON file

  - Parameters:

    - `filename`: (Required) Path to write the JSON file

    - `data`: (Required) Object to serialize and write

  - Returns: No return value



- **inspect(obj)** - Utility for inspecting object structures

  - `obj`: (Required) Object to inspect

  - Returns: String representation of the object structure



### cache.js



Caching utilities:



- **readFromCache(key)** - Read data from cache

  - `key`: (Required) Cache entry identifier

  - Returns: Cached data or null if not found



- **writeToCache(key, data)** - Write data to cache

  - Parameters:

    - `key`: (Required) Cache entry identifier

    - `data`: (Required) Data to store in the cache

  - Returns: No return value



## Environment Configuration



Place the commercetools connection parameters in a file named **.env** in a folder adjacent to your calling script or in the current directory.



Required environment variables:

```

CTP_PROJECT_KEY=your-project-key

CTP_CLIENT_SECRET=your-client-secret

CTP_CLIENT_ID=your-client-id

CTP_AUTH_URL=https://auth.region.commercetools.com

CTP_API_URL=https://api.region.commercetools.com

CTP_SCOPES=your-scopes

```



Optional:

```

LOG_API_CALLS=true

```



## Linking locally



If developing locally, do 'yarn link' in this directory, then 'yarn link @cboyke/demotools' in the dependent folder.