Most Recent

Thursday, March 2, 2017

ATG Endeca Integration : Interview Questions and Answers- Part2

ATG Endeca Integrations - Interview Questions - Part-2 1. What are basic components used to Integrate ATG and Endeca? 2. What are Acces... thumbnail 1 summary

    • ATG Endeca Integrations - Interview Questions - Part-2

     1. What are basic components used to Integrate ATG and Endeca?
    2. What are Accessors and how it helps in indexing? 
    3. Can Baseline/Partial Indexing triggered from ATG?
    4. Can attribute to make it as searchable from product-sku-output-config.xml?
    5. How to create a hierarchical dimension?
    6. What is CategoryTreeService and how it works?
    7. Why do we need Category Tree service?
    8. How to Add properties to Dimensions?
    9. Why CRS has extended RefinementMenuHandler that Endeca implementation can not support it?
    10. How to change the search interface name?
    11. What are different types of Accessors?
    12. How to test indexing using a Product record using dyn/admin?
    13. How ATG interact with Endeca to get content from Endeca Experience Manager?
    14. is it possible to add multiple hierarchical dimensions?
    15. What is the use of CAS Record store during Indexing? Do we always need record store for indexing without ATG integrations?
    Reply Your Answers to Comment Section. This would help others to understand Endeca Basics. I will consolidate the reply and post the answers. Will post another set of Questions soon.

    Monday, January 2, 2017

    Endeca Basics : Interview Questions and Answers- Part1

    Endeca Basics - Interview Questions - Part-1 1. What is Endeca and why do you use Endeca instead of other search products? 2. What are ... thumbnail 1 summary

    • Endeca Basics - Interview Questions - Part-1

     1. What is Endeca and why do you use Endeca instead of other search products?
    2. What are dimensions and properties? 
    3. What is difference between single and multi dimension and how to create?
    4. Can same property used for dimension and properties if possible than how?
    5. What is Hierarchical Dimension and how can you create?
    6. How can you get complete hierarchy of hierarchical dimension using single query?
    7. What is Dimension search and is it different from Record search?
    8. Which Endeca query Parameters and value gives all Endeca Records from MDEX Engine?
    9. is it possible to get Record and Dimension search using single query?
    10. What are different Types of Endeca Query Available?
    11. is it possible to store some properties inside dimension and how? Where do you use this scenario?
    12. What are different types of Dimension available in Endeca?
    13. What is difference between Internal and External Dimension and where dimension values gets stored(File)?
    14. How to modify the Dimension ID?
    15. How to remove some Dimension value(not used) from Endeca Experience manager?
    Reply Your Answers to Comment Section. This would help others to understand Endeca Basics. I will consolidate the reply and post the answers. Will post another set of Questions soon.

    Wednesday, December 14, 2016

    How to add user segment from ATG to Endeca witout creating segment in ATG?

    ATG passes all the user segments to endeca query based on profile to tigger different rule or content. There are some situations where som... thumbnail 1 summary

    • ATG passes all the user segments to endeca query based on profile to tigger different rule or content. There are some situations where some user segment does not exist in profile but required for endeca query to trigger different types of content.

     Example :- Show different banner if user is coming from affiliate A and show different banner if user is coming from affiliate B.

     This can be achived using below code to add user sgament based on different business logic
    public class ExtendNavigationStateProcessor implements NavigationStateProcessor {
      private UserState mUserState = null;
      @Override
    public void process(NavigationState pNavigationState) {

     if(Condition-1){
    getUserState().addUserSegments("Affliate-A");
    }
    else{
    getUserState().addUserSegments("Affliate-B");
    }
    }

    public UserState getUserState() {
    return mUserState;
    }

    public void setUserState(UserState pUserState) {
    mUserState = pUserState;
    }

    }


    From Author
    Was this post resolve the solution you are looking for? are you looking for some other issues?Provide your comments

    Wednesday, April 27, 2016

    How to Migrate Endeca Application between environment for forge-based Indexing?

    As part of the development/debugging phase, Endeca Application needs to be migrated from QA to UAT or Production to Staging. Following it... thumbnail 1 summary
    • As part of the development/debugging phase, Endeca Application needs to be migrated from QA to UAT or Production to Staging.

     Following items need to be migrated between environments. Find out the description below for each item.

    • Workbench content - Business creates content using Endeca Experience manager and gets stored in IFCR(InFront Configuration Repository)
    • State File - State file contains Endeca auto generated dimension values and FCM generated dimension value. Endeca N-value can be different if state files do not get copied to the environment.
    • Endeca Pipeline -  This Folder contains Endeca Application indexing configuration XML including properties, dimensions, search etc.
    • Cartridge Templates -  This folder contains all cartridge templates that is used to create content in Experience Manager.
    • Editors Config - This Folder contains configuration file to make a call to MDEX to show appropriate data in Experience manager cartridges.
    • Record Store - Record store contains all indexable records(Product catalog, content, record store etc.) and dimension information.

    Endeca Dimension Cartridge Editors
    Cartridge Editors
    As part of the migration process, the application needs to be exported first and the same set of files can be used to import into another environment.

     Export Endeca Application steps 

     1. Record Store - Click here to get more details. This Step can be optional if baseline triggers using /atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin Component

     2. State files - Copy <<Endeca_App>>/data/state folder.


     3. Endeca Pipeline - Copy <<Endeca_App>>/config/pipeline folder

    4. Cartridge Templates - Copy <<Endeca_App>>/config/cartridge_templates folder.

    5. Editors Config - Copy <<Endeca_App>>/config/editors_config folder

    6. Workbench Content - Navigate To Endeca Application control folder and run following commands to get export 
           <<Endeca_App>>/control->export_site.[bat|sh]
          This would generate the XML file and take the backup
    Endeca Export Site

    Import Endeca Application steps - Steps should be in the following order.

     1. Record Store - Click here to get more details. This Step can be optional if baseline triggers using /atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin Component.

     2. State files - Take the backup of <<Endeca_App>>/data/state folder and move state files copied from step-2  during export.

     3. Endeca Pipeline - Take the backup of <<Endeca_App>>/config/pipeline folder and move pipeline files copied from step-3  during export.

     4. Cartridge Templates - Copy cartridge_templates  from step 4 and run the following command to apply to Workbench. 
            <<Endeca_App>>/control -> set_templates.[bat|sh]
    Endeca Set Template command

    5. Editors Config - Copy editors_config from step 5 and change host and post information based on environment under each json file and run the following command
             <<Endeca_App>>/control -> set_editors_config.[bat|sh]

     6. Endeca Baseline Update-  Run below Command

          <<Endeca_App>>/control -> ./baseline_update.[bat|sh]

    7. Workbench Content - Import generated XML from Step-6 (Import site) to the Workbench content u. Run the following command to  import XM content
            <<Endeca_App>>/control -> ./import_site.[bat|sh]<<File_Name>>

    Endeca Import Site command
    8. Promote Content - Run the following command to promote content from Authoring to Live. Promote content can be run the Workbench as well.
           <<Endeca_App>>/control -> ./promote_content.[bat|sh]

    Endeca Promote Content
    From Author

    Was this post resolve the solution you are looking for? are you looking for some other issues?Provide your comments

    Sunday, April 24, 2016

    CAS Based Indexing : What is Index-config.json

    The Index Configuration modifies index configuration stored in the Endeca Configuration Repository for an application. This utility is typic... thumbnail 1 summary
    • The Index Configuration modifies index configuration stored in the Endeca Configuration Repository for an application. This utility is typically used to modify data after it has been exported from a product catalog system and modify the search configuration settings for the data.

    The following configuration can be done using index-config.json:

    • Endecaproperties,derived properties,and dimensions.
    • Precedence rules. These are specified under the precedenceRules node.
    • Search configuration. These are specified under the searchIndexConfig node.
    This file can be find out under <<Endeca_App>>/config/index-config/index-config.json

    Find out schema definition below 


    endeca:property 
    • isEnabled
      • Optional. Indicates whether the property is processed by CAS when CAS writes MDEX-compatible output. A value of true includes the property during processing; false excludes the property. This setting is useful when troubleshooting data issues for specific attributes. If omitted, the default value is true.
    • isRecordFilterable
      • Optional. Indicates whether the property can be used to filter records. Record filtering presents a subset of the data to the end-user. If omitted, the default value is false.
    • isRecordSearchEnabled
      • Optional. Specifies whether or not record search should be enabled for this property. Record search finds all records in an Endeca application that are tagged with an Endeca property that matches a term the user provides. You must enable each property that you want available for the record search. If omitted, the default value is false.
    • isRollupKey
      • Optional. Indicates whether the property can be used as a rollup key. This allows aggregated records to be based on this Endeca property. If omitted, the default value is false
    • isWildcardEnabledInRecordSearch
      • Optional. Indicates whether wildcard search is enabled for this Endeca property
    • mergeAction - ADD, UPDATE
      • Optional. The mergeAction specifies how to merge the attribute into the index configuration
    • propertyDataTYpe - ALPHA, INTEGER, DOUBLE, GEOCODE, DATETIME, DURATION, TIME
      • If omitted, the default value of propertyDataType is ALPHA
    • sourcePropertyName (multi-valued)
    endeca:derivedProperty
    • derivedPropertyFunction - MIN, MAX, SUM, AVG
    • derivedPropertySource
    • isEnabled
    • mergeAction
    endeca:dimension
    • displayOrder 
    • isAutogen
    • isEnabled
    • isHierarchicalDimensionSearchEnabled
    • isHierarchicalRecordSearchEnabled
    • isRecordSearchEnabled
      • Optional. Specifies whether or not record search should be enabled for this dimension
    • isWildcardEnabledInRecordSearch
    • mergeAction
    • multiSelectType - NONE, OR, AND
      • Optional. The multiSelectType enumerates the valid values for specifying multi-select dimensions
    • rangeComparisonType - STRING, INTEGER, FLOAT
      • Optional. This is used to create a range dimension
    • sourcePropertyName



    endeca:precedenceRules

    • isEnabled
    • isLeafTrigger 
    • mergeAction 
    • triggerDimension 
      • Required. Specifies the trigger dimension for a precedence rule.
    • triggerDimensionValueSpec
    • targetDimension
      • Required. Specifies the target dimension for a precedence rule
    Precedence_Rules_Index-config.json


    How to apply to Endeca Application

    Updated json can be set into application using below command


    <<Endeca_APP_PATH>>/control->./index_config_cmd.sh set-config -o system -f <<File_Path>>/index-config.json


    From Author

    Was this post resolve the solution you are looking for?Provide your comments

    Wednesday, April 20, 2016

    How to Remove a deployed Endeca application?

    Endeca Application is created using deployment template available under ToolsandFramework. Do not delete the application directory before... thumbnail 1 summary
    • Endeca Application is created using deployment template available under ToolsandFramework.

    Do not delete the application directory before removing to Endeca Application from EAC.

     Find out steps below to remove Endeca application

     1.   Remove the application from Endeca Application Controller(EAC)
      1. Run eaccmd.sh list-apps from any directory.
      2. /runcommand.sh --remove-app from application control folder as per the screenshot below
      3. Execute step-1 once again to verify if the application has been removed successfully or not.
    Endeca Application --remove-app


     2.   Go to apps directory and delete application directory.

     Note : Removing application does not remove DimensionValueIdManager and record store. Find out the screen shot below.


    From Author

    Was this post resolve the solution you are looking for? what do you think about this solution?Provide your comments

      Sunday, April 10, 2016

      Endeca Application : How to remove host entry from AppConfig files and EAC?

      AppConfig.xml import set of XML files related to host configuration, process definition etc. LiveDgraphCluster.xml contains host and port en... thumbnail 1 summary
      • AppConfig.xml import set of XML files related to host configuration, process definition etc. LiveDgraphCluster.xml contains host and port entry related to multiple Dgraph and this file is the potential candidate to add/remove host entry.

       Behaviour:-

       Suppose one new MDEX host added as per need basis and after sometimes extra MDEX host needs to be removed from LiveDgraphCluster.xml.

       <host id="LiveMDEXHostB" hostName="<<MDEXHost IP>>" port="8888" />

       Run below command. This command does not remove any host from EAC.

       ./runncommand.sh --update-definition
      Solution :- 

       --update-definition  can only add Dgraphs and MDEXHosts by updating the LiveDgraphCluster.xml  but not remove it.

       To remove MDEXHosts, below command needs to be run explicitly

       eaccmd remove-host  --app CRS --host localhost

      Remove host using eaccmd command

      Tuesday, April 5, 2016

      Endeca Basics :Does all MDEX/Dgraph feature configuration/XM changes required indexing?

      There are two categories of MDEX Engine feature configuration: 1. Dynamic configuration -   This can be updated in a running MDEX Engine wi... thumbnail 1 summary
      • There are two categories of MDEX Engine feature configuration:

      1. Dynamic configuration -  

      This can be updated in a running MDEX Engine without re-indexing or baseline process.
      • Search Interfaces
      • Thesaurus
      • Automatic Phrasing
      • Keyword Redirects
      • Rules


      2. Static configuration -  

      This needs to be specified as part of indexing process and the baseline is required.
      • Stop words
      • Precedence Rules
      • Dimension Ordering
      • Stemming
      Click here to get more topics related to Endeca basic concepts.

      Sunday, April 3, 2016

      Endeca Basics : Partial update Process Limitation

      A partial update modifies only those portions of the MDEX Engine index that have changed since the last baseline update. Partial updates can... thumbnail 1 summary
      • A partial update modifies only those portions of the MDEX Engine index that have changed since the last baseline update. Partial updates can affect any number of records in the system, and the update may be run frequently. A partial update does not require restarting the MDEX Engine.

       Find out the Partial update process limitation below :-


      • Add new dimensions.
      • Add new properties.
      • Add new mid-hierarchy dimension values in hierarchical dimension e.g Product.category.
      • Add, delete, or change any aspect of an existing dimension value. For example, you cannot add,change, or remove dimension value properties.
      • Add dimension value properties to any existing dimension values.
      • Update the index configuration files (such as the thesaurus and stop words files).
      Click here to get more topics related to Endeca basic concepts.

      Endeca Basics : How to troubleshoot partial update is working as expected or not?

      Endeca Partial indexing process uses only Forge process to generate XML file and Dgidx process gets skipped.   The MDEX Engine processes p... thumbnail 1 summary
      • Endeca Partial indexing process uses only Forge process to generate XML file and Dgidx process gets skipped.  

       The MDEX Engine processes partial updates concurrently with processing incoming queries. This function is also known as the continuous query. The partial update does not require restarting of Dgraph servers.
      Find out steps below how partial gets applied to MDEX(Multi Dimensional Index)

       1. Once XML gets generated by Forge process, MDEX engine process updates on Record-by-record basis.

       2. Generated XML from step 1 copied to <<Endeca/App>>/data/dgraphs/DgraphA1/dgraph_input/updates folder.

       3. Update fails or succeeded at the record level.

       4. File gets deleted under <<Endeca/App>>/data/dgraphs/DgraphA1/dgraph_input/updates folder once partial update gets applied to MDEX.

      If there any error occurs during partial update :-

       1. Partial update process continues to apply to MDEX.

       2. A log entry added to Dgraph logs

       3. MDEX engine writes a  file containing the failed updated record and dimension value under <<Endeca/App>>/data/dgraphs/DgraphA1/dgraph_input/updates/failed_update folder

      Click here to get more topics related to Endeca basic concepts.

      Endeca Basics : How to Debug Endeca Pipeline Components transformation is working as expected?

      A pipeline is composed of a collection of components. Each component performs a specific function during the transformation of  source data ... thumbnail 1 summary
      • A pipeline is composed of a collection of components. Each component performs a specific function during the transformation of  source data into Endeca records. Components are linked together by means of cross-references, giving the pipeline a sequential flow.

      Find out the way below to debug Endeca Pipeline Components transformation working as expected or not.

      1. Set the global log level. Log level can be set in Forge component using -v[f|e|w|i|d] option. Use -vd flag to increase Forge logging visibility as debug mode.

      2. -printRecords flag option to print records as they are produced by each pipeline component.

      3. Use Record Adapter - Insert a Record adapter between the pipeline components that needs to be review and set direction as Output. Make sure the Record source the component whose output needs to be examined. Pipeline stores the data in XML format and can be seen output XML under <<Endeca_App>>/data/processing folder.

      Find out screen shot below 
      Endeca Pipeline Output Record Adpater Debugging

      Tuesday, March 29, 2016

      Oracle Endeca : Change Default indexing order in Endeca Dgraph during baseline update

      Behaviour : - Records are getting indexed in Endeca and sort based on insertion order(FIFO) during baseline update from any source e.g re... thumbnail 1 summary
      • Behaviour : -

      Records are getting indexed in Endeca and sort based on insertion order(FIFO) during baseline update from any source e.g record store, JDBC query etc. Can indexing order be changed?

      Solution :- 

       Dgidx process reads data from either CAS or Forge process output and stores into the index. Dgidx provides a flag to change default insertion order.

       Add --sort <spec> into dgidx process. The format of <spec> is :"key|dir"

       Multiple sort keys can be specified as per below  format:
      "key_1[|dir_1]||key_2[|dir_2]||...||key_n[|dir_n]"

       Example :- --sort in_stock|desc||best_seller|desc

       Following entry needs to be added in DataIngest.xml under <Endeca_App>/config/script folder
      <dgidx id="Dgidx" host-id="ITLHost">
          <properties>
            <property name="numLogBackups" value="10" />
            <property name="numIndexBackups" value="3" />
          </properties>
          <args>
            <arg>-v</arg>
            <arg>--compoundDimSearch</arg>
            <arg>--sort in_stock|desc||best_seller|desc</arg>
          </args>
          <log-dir>./logs/dgidxs/Dgidx</log-dir>
          <input-dir>./data/cas_output</input-dir>
          <output-dir>./data/dgidx_output</output-dir>
          <temp-dir>./data/temp</temp-dir>
          <run-aspell>true</run-aspell>
        </dgidx> 




      Friday, March 25, 2016

      Oracle Endeca Commerce Architectural(XM and Assembler) Comparison Over Multiple Versions

      The Endeca Assembler and Experience Manager work together to provide the dynamic experience. Experience Manager  - Provides an interface... thumbnail 1 summary
      • The Endeca Assembler and Experience Manager work together to provide the dynamic experience.
      Experience Manager - Provides an interface to manage content configuration, enabling content administrators to create a targeted and compelling customer experience.
      The Endeca Assembler - Provides a unified interface for all content queries. The Assembler extracts information from the request and makes a query to the MDEX Engine to retrieve the configuration(Rule, Thesaurus, Stemming etc).
      Query Processing Flow - Application sends the request to the Endeca Assembler once the request comes from the user and Assembler fire a content query to MDEX to get the configuration to display a page. Point 2 and 3 are still valid for 11.2 version as thesaurus, stemming etc data is required for rule expansion.

      Here is the comparison cart below over multiple versions of Oracle Endeca releases. This Comparison talks about Endeca Content query response as per query process flow, daigram above, point 2 and 3. 
      Oracle Endeca versions performance comparison over multiple versions

      As per my analysis above, 11.2 version gives very good performance benifit in terms of Assember content query response time, size and remove application server dependency from IFCR repository.


      Click here to view Dgraph Performance Tuning


      From Author
      Was this post resolve the Endeca Performance issue? what do you think, making above steps may resolve most of the issues? Provide your comments 

      Thursday, March 24, 2016

      Spelling Auto Correction : how to disable Auto Correction for certain search term?

      Spelling auto-correction for record search automatically computes alternate spellings for user query terms that are misspelled. Spelling cor... thumbnail 1 summary
      • Spelling auto-correction for record search automatically computes alternate spellings for user query terms that are misspelled.

      Spelling correction is usually enabled in Dgraph flag "--spl" and this would enable spelling auto-correction for the record search. If there are requirements to disable auto-correction use, some extra parameters needs to be passed in Endeca MDEX Query as below :

       Ntx=mode+matchallany+spell+nospell 

      "spell+nospell" will disable spelling auto correction. This can be tested in jsp_ref implementation.

      Note:- Assembler implementation does not support above approach till current version.

      Sunday, March 20, 2016

      Oracle Endeca - How to create a dimension based on more than one attributes?

      In Endeca, A Dimension can be created using multiple properties. Example: More Ways to Shop dimension should contain 3 different proper... thumbnail 1 summary
      • In Endeca, A Dimension can be created using multiple properties.

      Example: More Ways to Shop dimension should contain 3 different properties sale, online, new so that user should be able to select multiple refinements under one dimension.

      Following are the ways to achieve it :-

      1.  Using Product-sku-output-config.xml : 

      Add outpur-name and use same name for all properties those needs to be combined for one Dimension
           <property name="online" is-dimension="true" type="string" multiselect-type="multi-or" output-name="product.moreWaysToshop"/>
          <property name="sale" is-dimension="true" type="float" multiselect-type="multi-or" output-name="product.moreWaysToshop"/>
          <property name="new " is-dimension="true" type="float" multiselect-type="multi-or" output-name="product.moreWaysToshop"/>



      2. Developer Studio:  

      Map dimension to multiple source properties as per screen shot below: - 

      Oracle Endeca Developer Studio



      3. Index-config.json - 

      This can be used for CAS based indexing where we have multiple source and joined using Endeca CAS system. use following examples to add mapping to index-config.xml and can be found under <<Endeca_App>>/config/mdex folder

       "product.moreWaystoShop" : {
       "sourcePropertyNames" : [ "new" ],
       "isAutogen" : true,
       "jcr:primaryType" : "endeca:dimension"
       },
       "product.moreWaystoShop" : {
       "sourcePropertyNames" : [ "online" ],
       "isAutogen" : true,
       "jcr:primaryType" : "endeca:dimension"
       },
       "product.moreWaystoShop" : {
       "sourcePropertyNames" : [ "sale" ],
       "isAutogen" : true,
       "jcr:primaryType" : "endeca:dimension"
       }


      Output would be looks like below in Endeca jspref   :-

      Multiple properties mapping to Dimension in Enddeca Jsp ref Implementation

      Endeca Basics : does Stemming change relevance ranking score?

      Stemming feature allows expanding the search results from singular to plural and vice versa. Suppose, search term is "shoe", this... thumbnail 1 summary
      • Stemming feature allows expanding the search results from singular to plural and vice versa.

       Suppose, search term is "shoe", this gives the result for "shoe" and "shoes" both.  Find out example below:-

      Record -A 
      DGraph.WhyDidItMatch: product.keywords: shoe (Exact,Substring/Phrase)
      DGraph.WhyDidItMatch: product.longDescription: Perfect for a day at the beach or around town. Though simple in design, this is a great everyday shoe with cork heel for extra walking comfort during daily wear. (Substring/Phrase)

      Record- B
      DGraph.WhyDidItMatch: product.keywords: shoes (Stemming)
      DGraph.WhyDidItMatch: product.longDescription: With a suit or with jeans, these shoes will be a hit in either case. Made of supple leather and with a rubber sole. (Stemming)

       Both above records show, why did it match. Record-A got match based on exact match and Record-B based on stemming.

       Relevance ranking of Endeca provides one of the modules "Interp", this module ranks the exact/phrase matched product always on top than stemming matched.

      Click here to get more topics related to Endeca basic concepts.

      Saturday, March 19, 2016

      Endeca Basics : How to Add/Delete word from stemming dictionary??

      Oracle Endeca provides a dictionary for stemming for all languages. This dictionary is part of MDEX engine installation. Dictionary can be ... thumbnail 1 summary
      • Oracle Endeca provides a dictionary for stemming for all languages. This dictionary is part of MDEX engine installation. Dictionary can be found under \MDEX\<<vesion>>\conf\stemming directory.

      Why do we need to delete stemming dictionary? One of the use case below:-

      A user searching for search term "short" displays "shorts" results as well and vice versa.

      There are two ways to Add/Delete word from stemming dictionary:-

      1. Open specific language file  under \MDEX\<<vesion>>\conf\stemming folder and edit. This is not the recommended way to do it.

      2. Modify the default stemming dictionaries by running Dgidx with the --stemming-updates flag and specifying an XML file that contains the updates to the dictionary that you want to make. The update file can include both additions and deletions. Dgidx processes the file by adding and deleting entries in the static stemming dictionary file.

      Adding entries to a static stemming dictionary

      Each entry in a static stemming dictionary is defined by an <ADD_WORD_FORMS> element and its sub-element <WORD_FORMS_COLLECTION>. For example, the following entry adds apple and its plural form apples to the static stemming dictionary:
      <!DOCTYPE WORD_FORMS_COLLECTION_UPDATES SYSTEM "word_forms_collection_updates.dtd">
      <WORD_FORMS_COLLECTION_UPDATES>
      <ADD_WORD_FORMS>
      <WORD_FORMS_COLLECTION>
      <WORD_FORMS>
      <WORD_FORM>apple</WORD_FORM>
      <WORD_FORM>apples</WORD_FORM>
      </WORD_FORMS>
      </WORD_FORMS_COLLECTION>
      </ADD_WORD_FORMS>
      </WORD_FORMS_COLLECTION_UPDATES>


      Deleting entries from a static stemming dictionary

      <!DOCTYPE WORD_FORMS_COLLECTION_UPDATES SYSTEM "word_forms_collection_updates.dtd">
      <WORD_FORMS_COLLECTION_UPDATES>
      <REMOVE_WORD_FORMS_KEYS>
      <WORD_FORM>aalborg</WORD_FORM>
      </REMOVE_WORD_FORMS_KEYS>
      </WORD_FORMS_COLLECTION_UPDATES>


      Combining deletes and adds

      You can also specify a combination of deletes and adds. Deletes are processed before adds. For example,the following XML removes aachen and then adds it and several stemmed variants of it.
      <!DOCTYPE WORD_FORMS_COLLECTION_UPDATES SYSTEM "word_forms_collection_updates.dtd">
      <WORD_FORMS_COLLECTION_UPDATES>
      <REMOVE_WORD_FORMS_KEYS>
      <WORD_FORM>aachen</WORD_FORM>
      </REMOVE_WORD_FORMS_KEYS>
      <ADD_WORD_FORMS>
      <WORD_FORMS_COLLECTION>
      <WORD_FORMS>
      <WORD_FORM>aachen</WORD_FORM>
      <WORD_FORM>aachens</WORD_FORM>
      <WORD_FORM>aachenes</WORD_FORM>
      </WORD_FORMS>
      </WORD_FORMS_COLLECTION>
      </ADD_WORD_FORMS>
      </WORD_FORMS_COLLECTION_UPDATES>


      Processing the update file

      To process the stemming update file, run Dgidx with the --stemming-updates flag and specify the XML file that contains the stemming updates.
      For example:
      dgidx --stemming-updates myAppStemmingChanges.en.xml

      Endeca Basics : How to Disable Endeca Rule Manager trigger for Stemming?

      Stemming enables keyword search results to provide results for both singular and plural words. Stemming and spelling correction both trigg... thumbnail 1 summary
      • Stemming enables keyword search results to provide results for both singular and plural words.

      Stemming and spelling correction both triggers Endeca Rules in Experience manager.


       Stemming can not be disabled for Rule triggering. However, if stemming needs to be disabled for few search term, remove those words from Stemming dictionary.

       Stemming Dictionary can be found under \MDEX\<<version>>\conf\stemming

      Endeca Basics : What is Stemming?

      The Stemming feature expands search results to include root words and variants of root words.Stemming enables words with a common root (such... thumbnail 1 summary
      • The Stemming feature expands search results to include root words and variants of root words.Stemming enables words with a common root (such as the singular and plural forms of nouns) to be used interchangeably by searches. 

       Stemming enables the MDEX engine to consider alternate forms of individual words as equivalent for the purpose of search query matching.

       For example
      search results for the word "shirt" will include the plural form "shirts", while a search for shirts will include the singular form shirt.

       Enable Stemming

       Stemming can be enabled as follow :-
      1. Forge based system -  Open CRS.stemming.xml under <<Endeca_App>>/config/pipeline folder
      2. CAS based system - Open CRS.stemming.xml under <<Endeca_App>>/config/mdex folder

       add the following snippet to enable for English:

       <STEMMING>
        <STEM_EN_US ENABLE="TRUE" />

      </STEMMING>

       Note:-  Endeca OOTB provides the stemming dictionary for many languages.


      ATG Endeca Integration : How to Troubleshoot Endeca Baseline update When Expected Data is Not Available in Endeca indices?

      Business creates data and push into Endeca for Indexing. Some type of data that was added through ATG and Endeca baseline update ran success... thumbnail 1 summary
      • Business creates data and push into Endeca for Indexing. Some type of data that was added through ATG and Endeca baseline update ran successfully without any errors, but the data changes are not reflected in the Endeca-based application.  

       Find out below to best way to troubleshoot the same. Issue can be either ATG end or Endeca end.

       1. Inspect the data/logs on ATG side

      By default, the following ATG components are involved to generate data that will be stored in Endeca CAS record stores for Product records. There are same set of Component for Content and Location. Follow the same for those as well incase any issues. 

      /atg/commerce/endeca/index/SchemaExporter
      /atg/commerce/endeca/index/CategoryToDimensionOutputConfig
      /atg/commerce/endeca/index/RepositoryTypeDimensionExporter
      /atg/commerce/search/ProductCatalogOutputConfig

      -Make sure there are no errors in the log for ATG server where baseline index is triggered.
      - Determine which of the above components is involved and check relevant data and its configurations.

      For SchemaExporter and RepositoryTypeDimensionExporter: Click on 'Show XML Output' link to check its data as per below sample URL

      /dyn/admin/nucleus/atg/commerce/endeca/index/RepositoryTypeDimensionExporter/?showXML=true

      For ProductCatalogOutputConfig or CategoryToDimensionOutputConfig:

       review its configurations as per below screen shot



      Verify Product data generated using ProductCatalogOutputConfig  component


      ProductCatalogOutputConfig component


      Verify Category data using CategoryToDimensionOutputConfig Component

      CategoryToDimensionOutputConfig Component

      CategoryToDimensionOutputConfig Component




       2. Inspect data/logs on Endeca end 

      Check if the data exists in Endeca after you confirm that the correct data is generated by ATG.

      Look for any errors in the following log files or files in the following directories.  

      On ITL Host where baseline runs: Foloowing are list of OOTB logs file.
      \Endeca\PlatformServices\workspace\log\process.{number}.log
      \Endeca\CAS\workspace\logs\cas-service.log
      \<app-name>\logs\<app-name>{number}.{number}.log, \dgidxs\Dgidx\Dgidx.log, \forges\ConfigurationGeneratorForge\ConfigurationGeneratorForge.log, \forges\Forge\Forge.log)

      Note : For any pipeline customization e.g. Manipulators, different data source ,verify appropriate logs for the same.

      On Dgraph host. For Development environment, Dgraph resides in ITL but in production arctitecture can be different:
      \<app-name>\logs\dgraphs\AuthoringDgraph
      \<app-name>\logs\dgraphs\DgraphA1

      Verify if the expected data available in record store or not.

      By default,  Endeca Deployment Template for Product Catalog Integration creates the following record stores.
      <app-name>_en_data
      <app-name>_en_schema
      <app-name>_en_dimvals

      Find out which record store holds the data that is not indexing in Endeca and extract data from the record store to verify if the data exists or not.

       Following commands read record store and convert data into xml file.
      <CAS_Installation_Path/version/bin>recordstore-cmd.bat|sh read-baseline -a <RecordStoreInstanceName> -f <output>.xml


       Verify data in following folders based on different systems.

       Forge-Based System - <App_Instalation_path>\data\forge_ouput.
      CAS-based system - <App_Instalation_path>\data\cas_ouput.

      Record data is in a binary file format. This data can extracted using $ENDECA_ROOT/<veriosn>/bin/Binary2xml script.
      Remaining file is in XML format to view.
      Example :-  $ENDECA_ROOT/<veriosn>/bin/Binary2xml <app-name>-sgmt1.records.binary output.xml

      Verify using endeca jsp reference application.

      Endeca jsp_ref application available in PlatformServices and ToolsAndFrameworks both.
      PlatformServices default port is 8888 and ToolsAndFrameworks is 8006.  Provide  'host' and 'port' in jsp ref text box to connect to Dgraph.

      http://<server>:<port>/endeca_jspref/controller.jsp

      Example:-

      http://<server>:8888/endeca_jspref/controller.jsp
      http://<server>:8006/endeca_jspref/controller.jsp



      It's your Turn
      Was this post help you to resolve the issue that you are looking for? Any other issues/topics that you might have and need a solution?

      Click here for more ATG-Endeca Integration related issue.


      Provide your feedback in the comment section below.

      Text Widget