Data Feed Requirements

This documentation describes the possibilities to provide your product data to excentos
and how to get the most out of your data.

In more complex situations, the docu helps you to make up a strategy
how to build up product data strategy and describes possibilities for data sourcing.

1

Intro &
Get Started

read on the basics of product data handling and what shall be done during project start. Check the data specification for the excentos Workbench.

2

Product
Attributes

see which product atttributes are required and how we distinguish general product attributes from category-specific product attributes

3

Data Formats
And Delivery

check how the data is delivered to excentos (e.g. FTP / HTTP) and which data formats such as CSV and XML we can handle

4

Data Sourcing Options

understand from which data sources the product data can come from

Introduction

The excentos Guided Selling-Technology contains powerful tools for data processing and data transformation, commonly known as extract, transform, load (ETL) processes. excentos' ETL tools allow to get the maximum out of your existing product data.

As Easy As Possible

We always strive to find solutions that minimize time and resources on your side to provide excentos with the data needed for the Product Guides. Our ETL tools can handle almost any data in every format from every database and typically uses your existing standard exports. Thus, the easiest way is that you just simply provide us with the data you currently have (see below for project start).

But Everything can be done

If providing your as-is-data is not „simply“ possible or if there are numerous options / complex databases, then read on in this documentation that describes the requirements, supported data formats and possible data setups for integrating product data into the Product Guides.

Project Start: Provide As-Is Test Data

At the beginning of the project or during the planing phase, please provide excentos with the data in the simplest way possible for you. If possible, provide us with as much data as you can (e.g. all sub-assortments, all available products) so that we can get an impression of all available data attributes.

What type of data does excentos need?

  • excentos needs all „core“ data / technical attributes / product description content
  • we need all pricing and availability data if you are a shop / website with dynamic assortment / availability
  • if you have separate data feeds from e.g. your Shop, your PIM (Product Information Management System) or a MAD (Media Asset Database), try providing all data sources to excentos if this is easily possible to you
The exact attributes are described in detail in the section product attributes below.

The excentos Matching Engine calculates product recommendations based on the user requirements (i.e. selections the visitor entered in the Product Guide). The Matching Engine infers the required product attribute (e.g. megapixel) from a user requirement (e.g. „high resolution images“). The product data thus does not need to contain user requirements since excentos infers user requirement-information from product data.
However if existing, we can further optimize the recommendation quality using your use case-oriented product data if available.

Example CSV Feed for the excentos Workbench

If you build your Product Guides with the excentos Workbench, there are 2 options:

  • you prepare the data feed so that it meets the data specification of the excentos Workbench
  • excentos prepares the data for you. Most of the times, we achieve a lot in very limited time since we have excellent data tools and technologies

Check out an example feed that represents the specification of the excentos Workbench data requirements:

Notes: The ODS file contains comments and highlights positive and negative examples. Open it in eg. Microsoft Excel or Libreoffice Calc. The CSV file is exported from the ODS file according to the following specification. Open it with any text editor and test its correctness in Excel or Calc:

TYPE / FIELD STATUS SPECIFICATION
file encoding suggestion CSV with encoding UTF-8
(other file encoding is supported, of course, and we recommend using an encoding that supports special characters)
quotes suggestion should be marked with '"' (double quotes)
(you are free to use different quotes and change the setting in the Workbench)
field separator suggestion needs to be a ',' (comma)
(you are free to use different quotes and change the setting in the Workbench)
text seperator suggestion '"' (double quotes)
(you are free to use different quotes and change the setting in the Workbench)
decimals seperator required need to be separated with a '.' (dot), e.g. '799.99'
units suggestion such as kilogram, currencies etc. need to be in a seperate field if you want your data field to be interpreted as a number
(we've marked this as suggestion only because the unit is only required if it shall be displayed in the Product Guide. The unit is not technically required, except if prices are displayed in the result list because we then need to know the currency)

Product Attributes

This section describes what Product Attributes excentos needs and why they are important.
There are General Product Attributes and Category-Specific Attributes.

General Product Attributes

General Product Attributes are identical across all product categories of your Shop / Website.

# ATTRIBUTE REQUIRED? DESCRIPTION
1 id Yes Your internal unique product ID (per unique product variant), can also be a common ID such as EAN or UPC
2 name Yes The name of the product to be displayed
3 image Yes A relative or absolute URL to the (largest) image(s) you have available on the product. excentos image server takes care of adjusting the images to the layout and resizes them to minimize payload. At least one image per product is required (or highly recommended); otherwise an image placeholder will be displayed. Please provide several images per product if possible. You can provide additional Attributes with more images for the product, if available.
4 price optional, Yes for retailer The current price of the product (1 quantity, calculated as you would display it in search result lists).
We can interpret several prices such as including / excluding VAT; before / after discount.
5 manufacturer Yes for Retailers The manufacturer name, required in a shop to display the brand
6 detailUrl only if you want to link to your product detail pages URL to your product detail pages. Typically used as outlink from the Product Guide to your shop / website
7 purchaseUrl optional URL to shopping basket. We recommend to include it since the product recommendation can then directly perform a "add to basket" action without the user needing to visit the product details page first.
8 variantId only if you have product variants Master variant ID of a product. Useful if your product assortment contains e.g. color or size variants and excentos can then filter down the result list to make sure only the master product is displayed.
The Product Guide will always recommend the best fitting variant among the products with the same Variants Master ID.
9 description recommended A short description text on the product. Can contain simple HTML formatting. To be displayed in the recommendation lists.
10 longDescription optional A long description text on the product (e.g. marketing text). Can contain simple HTML formatting. Can be displayed in the recommendation list, and can also be used by excentos to learn knowledge about your products by analyzing the text with machine learning algorithms.
11 currency unit optional Not strictly necessary, but cleaner. The currency can be directly included in the price attribute, too.
12 manufacturerProductID Only if we use third-party licensed data The ID given by the manufacturer to the product. This is the most important ID to be able mapping the product to a third-party database
13 Status / Available / Online etc optional If you can not restrict your product feed to products actually available, please provide a suitable filter attribute to excentos
14 Category optional Having a category indicator for the product is often helpful. Please try to include the tree of parent categories (in a separate attribute if possible)
15 Push factors / Merchandising information optional If you have information available that helps excentos to optimize the product listing for your business (such as margin, stock levels), include them.

Category-Specific Data

Category-Specific Attributes describe the product itself, such as technical data, possible use-cases and everything that distinguishes one product from the other. For every Product Guide, we need all the category specific attributes like e.g. the screen resolution of a notebook computer, the shoe size of a mountaineering boot, the range of height adjustment of an office desk etc. The more data we have, the more powerful and detailed the Product Guide can be.

The categorry-specific attributes can have the same data structure (mostly the case with CSV files) or a separate data structure / different attributes per category (mostly the case with XML data structures).

Category-Specific Attributes are not required for the data sourcing option „third-party licensed data“.

Example of Category-Specific Attributes

Typical CSV file with product data for, e.g., bicycles:


product data example csv raw

This is how the CSV file needs to look like if you open it in e.g. Excel if you have correctly set the text delimiters (in the example above we used a comma):

product data example csv table

Data Delivery and Data Formats

Delivery / Transfer of your Data

Typically you would provision the export feeds on an (S)FTP or an HTTP(S) server that's available from the public internet (we cannot access export feeds inside VPNs etc.). You can secure the server with username and password if required. Please always export the feed to the same filename and path and simply overwrite old versions. excentos will pull the file either periodically or frequently check for changes and pull the file only on demand. The preconfigured data update takes place nightly / hourly / on an agreed frequency, but depending on the frequency and criticality of your portfolio updates, we will agree on the best fitting update scheme for you. If necessary, excentos can provide an SFTP upload server, too.


General Data Format Tips

As mentioned above, excentos can process any data formats and thrives to use your existing data feed, so the following are only rule-of-thumb suggestions.

We typically use the standard exports of your Shop / eCommerce System / PIM / Product Database. Our recommendations are:

  • Character Encoding: UTF-8
  • try not to combine many individual attributes into one text attribute (we can parse values into their components, but that's not reliable)
  • export all values as explicit final values (e.g. „summer price list“ may be a precise definition of the price inside your system, but for us it's not very helpful in comparison to „39.00 $“)
  • when exporting number attributes, stick to one number format per attribute (i.e. don't mix decimal separators and unit positioning).

CSV Format

If done well, still the preferred and easiest format. The following tips can help a lot:

  • Escape control characters, tabulators and especially line breaks inside field values ( \n \t \“ \; etc.)
  • semicolon or comma are more robust field separators than tabulators
  • Either give everything text delimiters ( 'xxx' ; 'yyy' ; ) or nothing ( xxx ; yyy ; ). Using delimiters is better.
  • include a header row with the attribute names (ideally, use ASCII attribute names without spaces)

Custom XML Formats / Existing XML Formats

Nearly everything's possible but the effort can differ a lot. Please inquire individually.
Preferred XML Formats that minize the time to configure ETL transforms are „flat“ without hierarchies and cross references. Every product should be contained in a separate node and contain all attributes as sub-node.
Example of a well-structured XML:

product data example xml

Data Sourcing Options

There are different data sources / strategies how to get the data needed for the Product Guides. We will decide together after requirements analysis. The most common option (and what we strive for) is the existing data source:

SOURCE

DESCRIPTION

EVALUATION

existing data source e.g. your PIM, eCommerce-Solution or other (commercial or proprietary) data source containing a database great if database contains what we need (see required attributes below) can be technically complex to integrate depending on your databases
new data source e.g. a new database or CSV / Excel file maintained by you or excentos less ideal since new data needs to be produced adequate if the existing databases don't contain the required attributes and if they are inflexible (e.g. integrate new data attributes)
third-party licensed data e.g. a third-party provider of product data, such as Icecat, CNET, Etilize exists for almost any Consumer Electronics, IT and White Goods category and for some sports / living categories excentos has existing interfaces to some third party data providers
mixed models e.g. use existing data source for core attributes (such as product name, product images, price) and a new data source for product attributes such as weight sometimes the fastest and most effective & flexible approach, especially if we require further attributes not contained in your database and if your database is inflexible

Please keep in mind that any enhancements on product data shall be available for and will enhance the overal web experience and all product search / presentation functions on your website, no matter whether provided by excentos or not – so any improvements are generally highly welcome and supported across your entire organization.

Number & Granularity of Feeds for International Solutions

You can either provide individual feeds per market / country / etc. or tell excentos a filter criterion on a central feed from which excentos can decide in which market to include the product. If prices, product names and descriptions differ, we recommend to provide separate data feeds for each market (although it would be technically possible to contain all markets in one feed).

We are happy to hear from you!

Would you like to know more about excentos Product Guides, our Services and Guided Selling?
Then call us or write us an e-mail. We are happy to answer all your questions.

* input required