Centralpoint's NLS (Natural Language Search)

▶ Watch Video

Normal module submissions should be reflected in the remote index within 15 minutes, but larger updates like data transfers occur overnight using the scheduled task. For this to occur you must schedule the Natural Language Search job in your master console to run after transfers and before high traffic hours.

Algolia

Before beginning you must create an Algolia account and configure the Application ID and App Keys in the Natural Language Search module properties.

Amazon Kendra

Before beginning you must create an AWS account and configure the Amazon Kendra Access Key, Amazon Kendra Secret Key, Amazon Kendra Account Id, and Amazon Kendra Role in the Natural Language Search module properties. Create the index using the Centralpoint "Initialize" option in the "Execute" selection array. Do not create the index within Amazon Kendra's toolset.

By default, Amazon Kendra limits string lists to 10 values per attribute. This can affect system filters and we recommend you contact AWS support once your index is set up and ask for this value to be increased to 200. Once this change is made you can inform Centralpoint using the Amazon Kendra Maximum String List Size Per Attribute module property.

Amazon Kendra does not support the creation of facets via the API, so you must create them manually. Once the index has finished creating, go to the Data Management > Facet definition view in the Amazon Kendra console. Manually create the following facet exactly as listed:

AncestorAudienceIds: String list, Facetable
Anchor: String, Displayable
AudienceIds: String list, Facetable, Displayable
AutoNumber: Long, Displayable
AverageRating: Long, Displayable
CommentCount: Long, Displayable
CpNlsSplitMatchText: String, Displayable
DataCurrentLookupIds: String list, Facetable, Displayable
DataCurrentTitles: String list, Searchable
DataId: String, Facetable, Displayable
EndDate: Date, Facetable, Displayable
IndexDataId: String, Facetable, Displayable
IntegrationId: String, Displayable
IsEcommerce: String, Displayable
IsEnabled: String, Facetable, Displayable
ModuleId: String, Facetable, Displayable
ParentAudienceIds: String list, Facetable
RoleSystemNames: String list, Facetable, Displayable
SplitOn: String, Facetable, Displayable
StartDate: Date, Facetable, Displayable
Summary: String, Searchable, Displayable
Target: String, Displayable
TaxonomyIds: String list, Facetable, Displayable
TaxonomyNames: String list, Searchable
Title: String, Facetable, Searchable, Sortable
[SystemName]: String, Displayable Add this for each attribute in Attributes (attribute in the NLS module).

Please do not execute the Rebuild option with Amazon Kendra. If you wish to dump and repopulate the index you can use the Refresh option instead.

An attribute called DocumentExcerpt (cpsys_Attributes:DocumentExcerpt) is automatically initialized with the summary attribute if it exists and overwritten with the document excerpt returned by Amazon Kendra.

Kendra can be very slow to reflect changes applied in the module. I would give it a ½ hour before judging the results.

Setup

It is intended that a module is only used in the same index only once. You may get unexpected results if you select the same module with the same Remote Index Name in two different records. If you to create two objects in the remote index for the same data id you should either use attribute splitting or add an anchor to at least one of the rows to generate unique object ids.

Configuring Attributes

Amazon Kendra requires that you manually add a facet for each included attribute.

For the attribute to be accessible with a cpsys_Attributes: Column CpScript, the following steps must occur:

Within Kendra the facet created for the attribute must have Display checked.
Within the Centralpoint NLS record, the attribute model must have the Displayable property set to true

It is highly recommended that you only include the attributes that include content that makes sense to search. Do not use the "_ALL" option with Amazon Kendra.

[
    {
        "SystemName": "_ALL",
        "StripHtml": true,
        "MaxLength": 4000,
        "Displayable": false
    }
]

Attribute Splitting

Attribute splitting may be required if one attribute contains an extremely large amount of data. It allows you to split one attribute into multiple search results and configure the anchor and the title of each search results. If you want the rest of the attributes to be searchable, you should first create a separate record which contains allows the rest of the attributes to be searchable. Then, you can configure an attribute splitting record to split the large attribute as demonstrated below. The following example splits the large attribute on H elements using the contents of the H element in the title and linking directly to the H element when the search result is clicked. Now the 4000 character limit is applied to each H element instead of the entire attribute. This also improves search relevance by taking users directly to the content they're looking for. When CapatureIndex is -1, the the system will generate a query string variable that anchors to the index of the match instead of using a standard anchor.

[
    {
        "SystemName": "RTFEditor1",
        "StripHtml": true,
        "MaxLength": 4000,
        "Split": {
            "RegexPattern": "<h[1-3]{1}.*?>(.*?)</h[1-3]{1}>",
            "AnchorCaptureIndex": -1,
            "TitleCaptureIndex": 1,
            "TitleFormatString": "{1} ({0})",
            "IsMatchRemovedFromValue": false
        },
        "Displayable": false
    }
]

When records in Algolia are created from the Attribute Splitting model, the text matched from the RegEx Pattern will be stored in an attribute named CpNlsSplitMatchText. If the RegEx Pattern looks for HTML markup, the HTML is stripped. This will occur by default and does not require changes to the console configuration.

IsMatchRemovedFromValue - This boolean property is set to false by default. Setting this to true will remove the match found with the RegEx Pattern from the attributes.{propName} value stored in the Algolia record.

For example, if a heading tag is matched, <h2>Hello World</h2><p>Lorem ipsum dolor</p>, only the content following match will be set in the Algolia property. In this example, the RTFEditor1 property will exclude the matched heading and contain the following: <p>Lorem ipsum dolor</p>

Index Settings

Rebuilding this index will override any changes made manually using the remote index web app. The Rebuild option is not supported by Amazon Kendra. All index settings must be made manually in the Amazon Kendra console.

{
    "SearchableAttributes": [
        "title",
        "taxonomy.Name",
        "attributes"
    ],
    "AttributesForFaceting": [
        "filterOnly(audiences.audienceId)",         "filterOnly(audiences.parentAudienceId)",         "filterOnly(audiences.ancestors.audienceId)",         "filterOnly(endDate)",         "filterOnly(indexDataId)",         "filterOnly(isEnabled)",         "filterOnly(moduleId)",         "filterOnly(roles.systemName)",         "filterOnly(startDate)",         "filterOnly(taxonomy.taxonomyId)",         "filterOnly(dataCurrent.lookupId)",         "filterOnly(dataId)"      ]
}

Centralpoint

Centralpoint's Contextual Search provides powerful, customizable search capabilities across your website content. This guide outlines how to set up and configure the three available search options:

NLS Autocomplete
Advanced Searches (Website Searches)
Module Searches (Results Views)

Step 1: Create an NLS Configuration Record

First, set up a record in the NLS Configuration module to define your search behavior:

Navigate to Admin > NLS Configuration (important: updating module properties must be done from Admin > NLS Configuration, not NLS & AI > NLS Configuration)
Click Add New to create a new record
Select "Centralpoint" as the Type
Configure the following properties:
- Search Behavior: Choose between Flexible (finds all relevant matches even if some query words are missing) or Strict (only returns results containing every query word)
- Maximum Query Length: Sets character limit for search queries (default: 256)
- Maximum Terms: Limits number of distinct search terms processed (default: 10)
- Minimum Term Length: Sets minimum character length for search terms (default: 3)
- Scoring Parameters: Configure how results are ranked:
  - Phrase Relevance
  - Prefix Scoring
  - Position Scoring
  - Freshness Scoring
  - Comment Count Scoring
  - Average Rating Scoring
  - Popularity Scoring
- Synonym Generation: Enable/disable synonyms in search
- Spelling Suggestion Generation: Enable/disable spelling corrections
- Search Event Logging: Enable/disable tracking of search queries
- Snippet Generation Type: Controls how content snippets are generated
- Noise Words: Common words filtered out of search queries

Note: The default scoring parameters have been carefully tuned for optimal performance. Disable any scoring features that you are not using (e.g., if comments aren't used, disabling comment count scoring will improve search performance).

Note: Synonym generation uses the content in the NLS & AI > Synonyms module. Entering too many synonyms could cause a slowdown in performance.

Step 2: Configure Your Desired Search Implementation

Option 1: NLS Autocomplete

To enable autocomplete search functionality:

Navigate to the Script Generator module
Click Add New to create a new record
Set Type = "NLS Autocomplete"
Configure these key properties:
- Index: Select the NLS index to search against
- Search Results Module: Select modules that use the specified index. Ensure these modules are included in your NLS Configuration, otherwise they won't show data.
- Configure additional search display options as needed
Save the record, copy the CpScript, and place it where you'd like it to render

Note: The Search Results Sources attribute can be modified after submission, but may reset if the sources change via other attributes.

Option 2: Advanced Searches (Website Searches)

To configure site-wide search capabilities:

Navigate to System Pages > Search
Select Enabled under the Natural Language Search attribute
Select your previously created NLS Configuration record
Enter the attributes to highlight and snippet in search results
Configure additional search display options as needed

Option 3: Module Searches (Results Views)

To implement search within specific modules:

Navigate to the Module Designer module
Select the desired module
Go to the Results View (#2) tab
Select Enabled under the Natural Language Search attribute
Select your previously created NLS Configuration record
Enter the attributes to highlight and snippet in search results

Performance Considerations

Search Behavior: Flexible search typically performs faster but may return more varied results
Maximum Base Results: Affects initial query performance (default: 1000)
Count Cache Duration: Controls caching of count operations to reduce database load
Search Event Logging Delay: Sets minimum seconds between logging search events to prevent excessive database entries
Snippet Generation Type: Dynamic snippet generation could take up to 1/2 second per displayed record when content is very long

Auto-linking and Post-click Highlighting

Auto-linking and post-click highlighting features are only available when Snippet Generation Type is set to Dynamic. To enable these features:

In the module designer (for Module Searches) or System Pages > Search (for Advanced Searches), locate the attribute you want to highlight/snippet
Set InsertObjectIdElement='true' on the snippeted attribute
For auto-linking to the first block containing search terms, ensure "Disable Auto Link to First Block" is set to "No" in your NLS Configuration
For highlighting search terms on the destination page after clicking, ensure "Disable Post Click Highlighting" is set to "No" in your NLS Configuration
Make sure the oid query string variable and #oid are included in your search result links

Important Implementation Notes

You should turn off memory in the query string and use Snippetting and Highlighting instead on module/advanced searches using NLS.
Add [cp:scripting key='Column' name='Query?' formatStringEscape='true' /] to the URL.
For snippetting jump links to work you must:
- Make sure the descattr/snippet attribute is properly set up
- Ensure OID is in the query string
- Set up the snippeted attribute CpScript in the details view as follows: [cp:scripting key='Attribute' SystemName='RTFEditor1' insertObjectIdElement='true' /]

Advanced Configuration

For detailed scoring adjustments and custom implementations, refer to the complete property documentation in the Contextual Search configuration.

To maintain performance during periods of elevated user activity, we recommend the following manual adjustments to reduce system strain. These are listed in order of impact:

Disable Spelling Suggestions and Synonyms These features perform computationally intensive linguistic analysis and are the most resource-heavy. Disabling them will immediately reduce query processing time. Reduce Maximum Search Results Lowering the result cap from 1,000 to 250 significantly reduces post-query sorting, scoring, and rendering times.

Turn Off Engagement Scoring Disable features like comments, ratings, and popularity weighting. These rely on aggregated user behavior data and can delay result finalization.

Switch Search Behavior Mode to “Flexible” "Strict" search behavior enforces exact matches, which increases computational overhead. "Flexible" mode maintains user experience while enabling faster results.

Disable Position Scoring If enabled, position scoring analyzes the location of keywords within content. Disabling it removes another layer of scoring complexity.

📊 Estimated Impact: These changes can improve search response times by 40–60% during peak load, without affecting the relevance of core search results.