Custom Properties for Datasources

In this document, we discuss how to define custom properties for a datasource and how to upload them with each document.

What are custom properties?

Fundamentally, a custom property is any additional information that is attached to a document. These properties can be used in 3 different ways:

• Operators & Facets: Ability to filter documents for your search
• Rendering: Show useful information in the rendering of the search result
• Searchability and Ranking: Make the document more searchable and serve as a ranking signal

The above 3 use-cases are explained in respective sections below. But first, we explain how you can set up and upload custom properties for your datasource.

Setting up custom properties

Before custom properties can be uploaded for documents, we must configure the datasource to accept the specific custom properties. You need to define them under an object definition.

The configuration can be done in 2 ways:

1. Admin console ( recommended )
2. API calls to /adddatasource

Using admin console

1. From the app setup page in the admin console, go to object definitions.

2. Define your object (if you haven't already) and click on property definitions.

An object is any logical categorization of documents within your datasource (for example, documents/spreadsheets/slides can be objects for a drive datasource).

3. Configure the properties in the property definition modal.

Using API Calls

Here is a sample API call for the above example:

curl -X POST https://customer-be.glean.com/api/index/v1/adddatasource  \
{
   "name": “testDatasource”
   "objectDefinitions": [
        {
            "name": "Account",
            "displayLabel": "Account",
            "propertyDefinitions": [
                {
                "name": "priority",
                "displayLabel": "Priority",
                "displayLabelPlural": "Priorities",
                "propertyType": "TEXT",
                "hideUiFacet": false
                }
            ]
        }
   ]
 ... 
}

from glean_indexing_api_client.api import datasources_api
from glean_indexing_api_client.model.custom_datasource_config import CustomDatasourceConfig
datasource_api = datasources_api.DatasourcesApi(api_client)

datasource_config = CustomDatasourceConfig(
    name="testDatasource",
    object_definitions=[
        ObjectDefinition(
            name="Account",
            display_label= "Account",
            property_definitions=[
                PropertyDefinition(
                    name="priority",
                    display_label="Priority",
                    display_label_plural="Priorities",
                    property_type="TEXT",
                    hide_ui_facet=False
                    ...
                )
            ],
        )
    ],
    ...
)
try:
  datasource_api.adddatasource_post(datasource_config)
except indexing_api.ApiException as e:
  print('Exception when calling DatasourcesApi->adddatasource_post: %s\\n' % e)

Uploading custom properties

After you have defined the properties, you can set the properties values while uploading the document. A sample API call for /indexdocument(link) where we illustrate how this is done is:

curl -X POST https://customer-be.glean.com/api/index/v1/indexdocument \
{
   "document": [
        {
            "title": "Account 1",
            "id": "Account_1",
            "customProperties": [
                {
                    "name": "priority",
                    "value": "High",
                }
            ]
            ...
        }
    ... 
   ]
}

from glean_indexing_api_client.api import documents_api
from glean_indexing_api_client.model.index_document_request import IndexDocumentRequest
from glean_indexing_api_client.model.document_definition import DocumentDefinition
from glean_indexing_api_client.model.content_definition import ContentDefinition
from glean_indexing_api_client.model.user_reference_definition import (
    UserReferenceDefinition,
)
from glean_indexing_api_client.model.document_permissions_definition import (
    DocumentPermissionsDefinition,
)

request = IndexDocumentRequest(
    # DocumentDefinition has many fields, we show the usage of a few basic ones.
    document=DocumentDefinition(
        title='Account 1',
        id='Account_1',
        custom_properties=[
            CustomProperty(
                name='priority', 
                value='High'
            )
        ]
        ...
    )
    ...
)
documents_api = documents_api.DocumentsApi(api_client)
try:
    documents_api.indexdocument_post(request)
except indexing_api.ApiException as e:
    print("Exception when calling DocumentsApi->indexdocument_post: %s\n" % e)

In the above API call we set the value for the custom property Priority as High for the document Account_1.

Operators and Facets

What are operators?

Operators are a powerful way to filter information in Glean. In the above example we use the operators app: and owner: with query Glean. This tells Glean to only look for documents which match “Glean” and belong to app:drive and have owner:”Steve Smith”. This allows you to find the relevant information faster.

What are Facets?

Facets are special kinds of operators which are shown as visual filters in the Glean UI. You can find them on the right hand side of the search result page. This removes the burden of remembering the operators and gives you a nice and interactive way of filtering results.

Example

Once you follow the steps above in defining custom properties and uploading them for a document, you should be able to use the priority operator and see it as a facet!

Rendering

Apart from the above mentioned steps for configuring and uploading custom properties, if you would like to show custom properties in the search result rendering - the following 2 steps are required:

1. In the UI options for the custom property, select Search Result (screenshot below) (or) set uiOptions for the custom property in the API call to SEARCH_RESULT .

2. Contact Glean support to show the custom property to the search result as per your preference. An example is shown below.

For more details on rendering of search result refer here

Searchability and Ranking

All custom properties are automatically used to enrich the document in the index - thereby improving searchability and ranking.

For example, if we have a ticketing datasource and include a ticket ID as a custom property - we can then search by the ticket ID to fetch the relevant ticket (without requiring it to be in the title or the body!).