You already have a mix of document IDs and URLs, and you want the indexed documents you can open without one request per document. One experimental batch accepts 1 to 100 document IDs or URLs. It prints the documents Glean returns under results, and the document IDs or URLs it does not return under warnings. The published TypeScript API client does not include these endpoints, so this example calls the HTTP API and uses the official auth package for sign-in.
You sign in with @gleanwork/auth and call the experimental document endpoints over HTTP. The published TypeScript API client does not include these endpoints. client.documents is a different API.
Post a mixed batch
Find your Glean backend, take a documents:read token from the auth package, and send X-Glean-Include-Experimental: true on the request. POST /api/documents/batch with 1 to 100 document IDs or URLs. Each item is a document_id or a url, not both. include CONTENT is a body field, not a query parameter.
import { loadEnvFile } from 'node:process';
import { createGleanTokenProvider, discoverGleanTenant } from '@gleanwork/auth';
import {
assertBatchCoverage,
batchRequest,
readDocumentsBatchResponse,
type DocumentSpec,
type DocumentsBatchResponse,
} from './documents.js';
import {
DocumentsTimeoutError,
documentsApiError,
readProblemDetail,
} from './errors.js';
// These experimental document endpoints are not in @gleanwork/api-client.
// client.documents is a different API. Call POST /api/documents/batch with
// fetch. Sign in with @gleanwork/auth. Every request sends
// X-Glean-Include-Experimental: true. A 404 with that header usually means
// your Glean instance has not turned the endpoints on. Ask your Glean admin
// or Glean support.
export const DOCUMENTS_SCOPE = 'documents:read';
export const EXPERIMENTAL_HEADER = 'X-Glean-Include-Experimental';
const REQUEST_TIMEOUT_MS = 30_000;
const LOOPBACK_HOSTS = new Set(['127.0.0.1', '::1', 'localhost']);
export interface GleanClientTarget {
email?: string;
serverUrl?: string;
}
export interface DocumentsClient {
origin: string;
batchGet(
specs: readonly DocumentSpec[],
includeContent: boolean,
): Promise<DocumentsBatchResponse>;
}
function loadDotEnv() {
try {
loadEnvFile();
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error;
}
}
export function assertBackendOrigin(serverURL: string): string {
let server: URL;
try {
server = new URL(serverURL);
} catch {
throw new Error('Use a complete Glean backend HTTPS origin.');
}
const loopback = LOOPBACK_HOSTS.has(server.hostname);
if (
(server.protocol !== 'https:' && !loopback) ||
server.username ||
server.password ||
server.search ||
server.hash ||
(server.pathname && server.pathname !== '/') ||
(!loopback && server.port)
) {
throw new Error('Use a complete Glean backend HTTPS origin.');
}
return server.origin;
}
async function resolveServerUrl({ email, serverUrl }: GleanClientTarget) {
const explicit = serverUrl?.trim();
if (explicit) return explicit;
const workEmail = email?.trim();
if (workEmail) return (await discoverGleanTenant(workEmail)).serverUrl;
const configured = process.env.GLEAN_SERVER_URL?.trim();
if (configured) return configured;
throw new Error(
'Pass --email or --server-url, or set GLEAN_SERVER_URL in your environment.',
);
}
function parseJson(text: string): unknown {
if (!text) return undefined;
try {
return JSON.parse(text) as unknown;
} catch {
return undefined;
}
}
function isTimeout(error: unknown): boolean {
return (
typeof error === 'object' &&
error !== null &&
'name' in error &&
error.name === 'TimeoutError'
);
}
export async function createDocumentsClient(
target: GleanClientTarget,
log: (message: string) => void = () => undefined,
): Promise<DocumentsClient> {
loadDotEnv();
const origin = assertBackendOrigin(await resolveServerUrl(target));
const staticToken = process.env.GLEAN_API_TOKEN?.trim();
if (staticToken) log('Using GLEAN_API_TOKEN from the environment.');
else log('Using the OAuth session (documents:read).');
const getAccessToken = staticToken
? () => Promise.resolve(staticToken)
: createGleanTokenProvider({
serverUrl: origin,
scopes: [DOCUMENTS_SCOPE],
});
async function request(
url: URL,
init: {
method?: string;
body?: string;
headers?: Record<string, string>;
} = {},
): Promise<unknown> {
const accessToken = await getAccessToken();
let response: Response;
try {
response = await fetch(url, {
method: init.method,
body: init.body,
redirect: 'error',
signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),
headers: {
accept: 'application/json',
...init.headers,
authorization: `Bearer ${accessToken}`,
[EXPERIMENTAL_HEADER]: 'true',
},
});
} catch (error) {
if (isTimeout(error)) throw new DocumentsTimeoutError();
const message = error instanceof Error ? error.message : String(error);
throw new Error(`Could not reach Glean: ${message}`);
}
const body = parseJson(await response.text());
if (!response.ok) throw documentsApiError(response, body);
if (readProblemDetail(body)) {
throw new Error('Document response was an error document with HTTP 200.');
}
return body;
}
return {
origin,
async batchGet(specs, includeContent) {
const payload = batchRequest(specs, includeContent);
const response = readDocumentsBatchResponse(
await request(new URL('/api/documents/batch', origin), {
method: 'POST',
body: JSON.stringify(payload),
headers: { 'content-type': 'application/json' },
}),
includeContent,
);
assertBatchCoverage(payload.document_specs.length, response);
return response;
},
};
}
Read results and warnings
Documents Glean returns are results. A document ID or URL Glean does not return is a warning on the same HTTP 200, with a code, a message, and context.index. Every item appears in exactly one of those lists. A warning does not fail the command. You still see Retrieved N of M documents. An HTTP error fails the whole batch.
import type { DocumentsClient } from './client.js';
import type { DocumentSpec, DocumentsBatchResponse } from './documents.js';
/**
* Documents Glean returns are printed from `results`. A document ID or URL
* Glean does not return is printed under `warnings` on the same HTTP 200,
* matched by `index`. The command still exits 0. If the HTTP call fails,
* the command fails. If you asked for extracted text and Glean cannot read
* it, the command fails instead of leaving `content` out.
*/
export async function batchRetrieveDocuments(
client: DocumentsClient,
input: { specs: readonly DocumentSpec[]; includeContent: boolean },
): Promise<DocumentsBatchResponse> {
return client.batchGet(input.specs, input.includeContent);
}
Copy the project onto your machine
Copy the runnable TypeScript CLI and its offline tests into a new directory. Sign-in uses the official @gleanwork/auth package.
npx -y tiged@2.12.8 gleanwork/glean-cookbook/recipes/batch-retrieve-documents batch-retrieve-documents
Install dependencies
Enter the project directory and install its dependencies. Run the remaining commands from this directory.
cd batch-retrieve-documents && npm install
Run the offline tests
Run the tests. They do not use your Glean login or the network. Passing tests does not prove you can reach your Glean instance.
npm test
Choose an authentication path
OAuth is the default. @gleanwork/auth discovers your instance, registers the OAuth client dynamically, and stores credentials outside the project. If OAuth is unavailable, copy .env.example to .env, set GLEAN_SERVER_URL to your backend origin, for example https://<instance>-be.glean.com, and create a user-scoped API token with Read documents (documents:read). On the legacy token screen, choose DOCUMENTS. Put that token in GLEAN_API_TOKEN and skip the next step. Omit --email from the remaining commands. If email discovery finds the wrong instance, replace --email with --server-url and that same backend origin.
Sign in with OAuth
Run the login command with the documents:read scope and finish authorization in your browser. Wait until the command reports success. Skip this step when you configured a token in .env.
npm run login -- --email "<work-email>"
Verify against your instance
Pass one URL you can open. This command prints Retrieved N of M documents, then request_id, results, and warnings. Copy the document ID in parentheses on the results line. Pass it to --id in the next command. With a token in .env, omit --email.
npm start -- --email "<work-email>" --url "<document-url>"
Include a URL that is not indexed, and extracted text
Repeat the URL, pass the document ID from the previous results line to --id, add a URL that is not an indexed document, and pass --content. Each result line includes content_segments: N. content_segments: 0 means there is no extracted text, and you still see the document. A document ID or URL Glean does not return is listed under warnings on HTTP 200, and the command still exits 0. The warning code and message are what Glean returns. With a token, omit --email.
npm start -- --email "<work-email>" --url "<document-url>" --id "<document-id>" --url "https://example.invalid/not-indexed" --content
This client sets X-Glean-Include-Experimental: true on every request. Keep that header if you add a call. A 404 with the header usually means your Glean instance has not turned these endpoints on. Ask your Glean admin or Glean support.
OAuth login and the token provider use the documents:read scope. Do not add other scopes. If you create an API token instead, choose Read documents (documents:read). On the legacy token screen, choose DOCUMENTS.
HTTP 200 with warnings means Glean finished the batch and did not return some document IDs or URLs. You see Retrieved N of M documents, then results and warnings, and the command exits 0. Exit non-zero only when the request itself fails, including HTTP 400, 401, 403, 404, and 413.
These endpoints are experimental and can change. This client sends X-Glean-Include-Experimental: true on every request. If your Glean instance has not turned the endpoints on, you still get HTTP 404. Ask your Glean admin or Glean support. The published @gleanwork/api-client package has no methods for these endpoints. client.documents reads documents through a different API. Do not call it here.
A Gmail or Outlook search result or chat citation can show an ID that only works in that result. Do not pass that ID to --id. Pass the message URL with --url. That URL can still appear under warnings with code resource_not_found. Go links and federated search results cannot be retrieved. You only get documents you can open.
Each document ID or URL appears under results or under warnings. The warning shows a code and Glean's message. resource_not_found and invalid_parameter are examples of codes Glean may return. Glean can return other codes. There is no separate status field on each document. An empty or malformed URL is listed under warnings on HTTP 200. It is not HTTP 400. This CLI rejects a blank --url or --id before it sends the request, so you see an error that names the flag. HTTP 413 means the request body is too large. Sending more than 100 document IDs and URLs is rejected in this CLI before the request, and that error is not HTTP 413.
include CONTENT adds content.full_text_list on every document Glean returns. An empty list means Glean stored no extracted text. You still see the document, with content_segments: 0. If Glean cannot read the text you asked for, the command fails instead of printing documents without content. The command prints the segment count, not every segment.
- Pass --content when you need extracted text. Each result line includes content_segments: N. content_segments: 0 means Glean stored no extracted text. You still see the document. If Glean cannot read the text you asked for, the command fails.
- Copy the document ID in parentheses on the results line and pass it to --id. You can also copy the document_id line from retrieve-document-by-url.
- For a Gmail or Outlook message, pass the message URL with --url, not the ID from the search result or chat citation.
Retrieve one document URL, then reuse its printed document ID with a URL that is not indexed
Prints Retrieved N of M documents, then request_id, results, and warnings. The results line shows the document ID in parentheses. A document ID or URL Glean cannot resolve is listed under warnings on HTTP 200, and the command exits 0. HTTP errors, including 400, 404, and 413, exit non-zero and do not print the Retrieved line. A 404 usually means your Glean instance has not turned these endpoints on. Ask your Glean admin or Glean support. With --content, each result line includes content_segments: N. content_segments: 0 means there is no extracted text, and you still see the document.
Run the authenticate step on this page. It discovers your tenant from work email and signs you in with OAuth, using the shipped login command. If OAuth is unavailable, create a scoped Glean-issued token in Token Management (documents:read).