Most Recent

Monday, January 2, 2017

Endeca Basics : Interview Questions and Answers- Part1

Endeca Basics - Interview Questions - Part-1 1. What is Endeca and why do you use Endeca instead of other search products? 2. What are ... thumbnail 1 summary

    • Endeca Basics - Interview Questions - Part-1

     1. What is Endeca and why do you use Endeca instead of other search products?
    2. What are dimensions and properties? 
    3. What is difference between single and multi dimension and how to create?
    4. Can same property used for dimension and properties if possible than how?
    5. What is Hierarchical Dimension and how can you create?
    6. How can you get complete hierarchy of hierarchical dimension using single query?
    7. What is Dimension search and is it different from Record search?
    8. Which Endeca query Parameters and value gives all Endeca Records from MDEX Engine?
    9. is it possible to get Record and Dimension search using single query?
    10. What are different Types of Endeca Query Available?
    11. is it possible to store some properties inside dimension and how? Where do you use this scenario?
    12. What are different types of Dimension available in Endeca?
    13. What is difference between Internal and External Dimension and where dimension values gets stored(File)?
    14. How to modify the Dimension ID?
    15. How to remove some Dimension value(not used) from Endeca Experience manager?
    Reply Your Answers to Comment Section. This would help others to understand Endeca Basics. I will consolidate the reply and post the answers. Will post another set of Questions soon.

    Sunday, April 24, 2016

    How to migrate CAS record store schema with data between environments?

    As per ATG-Endeca Integration, All product catalog data gets stored in Endeca CAS record store during baseline process. Sometimes, these da... thumbnail 1 summary
    • As per ATG-Endeca Integration, All product catalog data gets stored in Endeca CAS record store during baseline process. Sometimes, these data needs to be migrated from Production to UAT or UAT to Dev.

     Find out steps below to Migrate.

     Export Data from Environment - Production

     1. Identify record store names to be migrated.

    Navigate to CAS Installation directory and list the record stores
    /CAS/<<Version_number>>/bin->./component-manager-cmd.sh list-components
    CRS_en_dimvals
    CRS_en_data
    CRS_en_schema

    2. Identify if there are any active-transactions or not

    /CAS/<<Version_number>>/bin->
    ./recordstore-cmd.sh list-active-transactions -a CRS_en_dimvals
    ./recordstore-cmd.sh list-active-transactions -a CRS_en_data
    ./recordstore-cmd.sh list-active-transactions -a CRS_en_schema

    3. Fetch the current record store configuration

    /CAS/<<Version_number>>/bin->

    ./recordstore-cmd.sh get-configuration -a CRS_en_dimvals -f CRS_en_dimvals.xml
    ./recordstore-cmd.sh get-configuration -a CRS_en_data -f CRS_en_data.xml
    ./recordstore-cmd.sh get-configuration -a CRS_en_schema -f CRS_en_schema.xml

    4. Read data from record store and write into XML files

    /CAS/<<Version_number>>/bin-> 
    ./recordstore-cmd.sh read-baseline -a CRS_en_dimvals -f data_CRS_en_dimvals.xml
    ./recordstore-cmd.sh read-baseline -a CRS_en_schema -f data_CRS_en_schema.xml
    ./recordstore-cmd.sh read-baseline -a CRS_en_data -f data_CRS_en_data.xml



    Import Data with schema to enviornment -UAT

    1. Set the CAS record Store Configuration using step 3 during import process

    /CAS/<<Version_number>>/bin-> 
    recordstore-cmd.bat set-configuration -a CRS_en_dimvals -f CRS_en_dimvals.xml
    recordstore-cmd.bat set-configuration -a CRS_en_schema -f CRS_en_schema.xml
    recordstore-cmd.bat set-configuration -a CRS_en_data -f CRS_en_data.xml

    2. Use step 4 data and write record into record store 
    /CAS/<<Version_number>>/bin-> 
    recordstore-cmd.bat write -a CRS_en_dimvals -f data_CRS_en_dimvals.xml
    recordstore-cmd.bat write -a CRS_en_schema -f data_CRS_en_schema.xml
    recordstore-cmd.bat write -a CRS_en_data -f data_CRS_en_data.xml

    Note :- CRS_en_schema is not used for CAS based Indexing after 11.1 version.

    Sunday, April 3, 2016

    Oracle Endeca 11.x : How Last Mile Crawl is used?

    Endeca baseline update process invokes last mile crawl to create Dgidx-compatible data and passes it to Dgidx to generate binary files for ... thumbnail 1 summary
    • Endeca baseline update process invokes last mile crawl to create Dgidx-compatible data and passes it to Dgidx to generate binary files for MDEX engine.

     Find out following operations during last-mile-crawl 

    • Merges index-config.json using system user and ATG user 
    • Merge Multiple record store defined in crawl configuration XML file.
    • Processes product/article/store records. Data manipulation can be done if required using custom manipulators.
    • Writes the schema and records in the MDEX-compatible format.

    How Last Mile gets created

     Endeca Application instance needs to create using deployment template. Following commands create last-mile-crawl

     <<Endeca_App>>/control/initialize_services.sh
    In turn the initialize_services.sh runs a following command
    ${CAS_ROOT}/bin/cas-cmd.sh createCrawls -h ${CAS_HOST} -p ${CAS_PORT} -f ${WORKING_DIR}/../config/cas/last-mile-crawl.xml 

     Features

     1.  Record Store Joins
    Where the <<Endeca_App>>/config/cas/last-mile-crawl.xml sets up the CAS Crawl with the names of the CAS Recordstores,
    <moduleProperties>
    <moduleProperty>
    <key>dataRecordStores</key>
    <value>CRS-data</value>
    <value>CRS-External-data</value>
    </moduleProperty>
    <moduleProperty>
    <key>dimensionValueRecordStores</key>
    <value>CRS-dimvals</value>
    </moduleProperty>
    </moduleProperties> 

     As per XML snapshot above, multiple record stores can be added for further processing. CAS Based indexing only support switch join between multiple Record store.

    2. Add Manipulators
    Java manipulators can be added into last-mile-crawl in case any data manipulation required e.g. remove comma and create multi-valued properties
    <manipulatorConfig>
    <moduleId>
    <id>com.endeca.cas.extension.sample.manipulator.
    substring.SubstringManipulator</id>
    </moduleId>
    <moduleProperties>
    <moduleProperty>
    <key>sourceProperty</key>
    <value>Endeca.Document.Text</value>
    </moduleProperty>
    <moduleProperty>
    <key>targetProperty</key>
    <value>Short.Truncated.Text</value>
    </moduleProperty>
    <moduleProperty>
    <key>length</key>
    <value>20</value>
    </moduleProperty>
    </moduleProperties>
    <id>Create short truncated text property</id>
    </manipulatorConfig>



    3. Merges index-config
    initialize_services.sh runs the following command to update the Endeca Configuration repository with the properties and dimensions mentioned in the ./index-config.json
    "${WORKING_DIR}/index_config_cmd.sh" set-config -f "${WORKING_DIR}/../config/index_config/index-config.json" -o all


    It's your Turn

     Was this blog helpful for you? What do you think about this post? Any other topics that you want to cover in details. 

     Provide your valuable comments or response below.

    Endeca Basics : How to Debug Endeca Pipeline Components transformation is working as expected?

    A pipeline is composed of a collection of components. Each component performs a specific function during the transformation of  source data ... thumbnail 1 summary
    • A pipeline is composed of a collection of components. Each component performs a specific function during the transformation of  source data into Endeca records. Components are linked together by means of cross-references, giving the pipeline a sequential flow.

    Find out the way below to debug Endeca Pipeline Components transformation working as expected or not.

    1. Set the global log level. Log level can be set in Forge component using -v[f|e|w|i|d] option. Use -vd flag to increase Forge logging visibility as debug mode.

    2. -printRecords flag option to print records as they are produced by each pipeline component.

    3. Use Record Adapter - Insert a Record adapter between the pipeline components that needs to be review and set direction as Output. Make sure the Record source the component whose output needs to be examined. Pipeline stores the data in XML format and can be seen output XML under <<Endeca_App>>/data/processing folder.

    Find out screen shot below 
    Endeca Pipeline Output Record Adpater Debugging

    Text Widget