Tabscanner for AI agents

Overview

Tabscanner turns a photo of a receipt into JSON: merchant, date, totals, taxes, tip, payment method and line items, each with a confidence score. The HTTP API is two calls: an upload that returns a token, and a result endpoint that is polled until the data is ready. See the API reference for every field.

For an agent, hide the polling. Expose one tool, scan_receipt, that takes an image and returns the result object, plus a trivial get_tabscanner_account. The model should never have to decide when to poll again, and it should never see the API key.

Everything the model needs to know about the data is in the field descriptions of openapi.yaml. If your agent will call the HTTP API directly, put the AGENTS.md snippet on the right in its instructions.

## Tabscanner receipt API (v2)

Use these rules when calling https://api3.tabscanner.com directly.

- Send `Authorization: Bearer $TABSCANNER_API_KEY`. Read the key from the environment; never print or log it.
- Upload: `POST /v2/receipts`, `multipart/form-data`, one part named `file` (JPEG or PNG). Add `region` (two-letter lower-case country code) when known. Send a new `Idempotency-Key: <uuid>` header per image.
- `202` means accepted: keep `token`. Any other status is an error: read `code` and `detail` from the JSON body, report them, and stop.
- Poll `GET /v2/receipts/{token}`, sleeping the `Retry-After` header's seconds before each call. `status` is `pending` (keep polling), `done` (data in `result`) or `failed` (unreadable image; do not retry it).
- Stop polling after 60 seconds (120 with a `model`) and report a timeout. The token stays valid.
- Retry network errors, `5xx`, `409` and `429` with backoff, reusing the same `Idempotency-Key` for uploads. Never retry other `4xx` unchanged.
- Each accepted upload uses one credit. Never re-upload an image to "check" it; fetch the result again with its token instead.
- `model: "harajuku"` works on every account. Only send a `-pro` model when `GET /v2/account` reports `features.proModels: true`; otherwise it fails with `403 MODEL_NOT_PERMITTED` (not charged).
- Ignore result fields you do not recognise.

Tool definitions

The JSON on the right defines both tools as a name, a description and a JSON Schema for the input. This is the shape every major agent framework and function-calling API accepts; if yours names the schema key differently (parameters instead of input_schema, for example), rename that key and change nothing else.

The input_schema properties other than image_path map one to one onto the upload's form fields, using snake_case: region, document_type (documentType) and model.

Design notes

• image_path is a local path because the agent runtime usually has the file. If your runtime passes URLs or base64, adapt the implementation, not the schema the model sees.
• Keep region optional. If the model knows the country from context (a user in the UK, a receipt in euros) it will pass it; forcing it invites guesses.
• Leave model optional and let the agent reach for it when standard OCR returns low confidences or broken arithmetic. See Choosing a model.
• Return the whole result object. Trimming it "to save tokens" removes the confidence fields the model needs to answer honestly.
• Surface errors as tool errors that include the API's code and detail, so the model can tell the user "out of credit" rather than "something went wrong".

[
  {
    "name": "scan_receipt",
    "description": "Extract structured data from a photo of a receipt or invoice using the Tabscanner API. Uploads the image, waits for processing (usually a few seconds; 10 to 20 seconds when a model is chosen) and returns the result object: establishment, date, total, subTotal, tax, tip, discount, currency, paymentMethod, address, lineItems, summaryItems and confidence scores (0 to 1). Each call uses one credit, so call it once per image. Pass the receipt's country as region for best accuracy. Before relying on total, check totalConfidence and that the lineItems add up.",
    "input_schema": {
      "type": "object",
      "properties": {
        "image_path": {
          "type": "string",
          "description": "Local path to a JPEG or PNG photo of the receipt."
        },
        "region": {
          "type": "string",
          "description": "Two-letter lower-case ISO 3166-1 country code of the receipt, for example gb, us or de. Omit if unknown; do not guess.",
          "pattern": "^[a-z]{2}$"
        },
        "document_type": {
          "type": "string",
          "enum": ["receipt", "invoice", "auto"],
          "description": "Type of document. Default receipt. Use auto when unsure."
        },
        "model": {
          "type": "string",
          "enum": ["harajuku", "harajuku-pro"],
          "description": "Optional Tabscanner model for difficult documents: harajuku reads more thoroughly than standard OCR; harajuku-pro is more thorough again and slower, and needs the Pro package (features.proModels). Omit for standard OCR, which suits most receipts. Without the Pro package a -pro model fails with MODEL_NOT_PERMITTED and nothing is charged."
        }
      },
      "required": ["image_path"],
      "additionalProperties": false
    }
  },
  {
    "name": "get_tabscanner_account",
    "description": "Return the remaining Tabscanner credit and the features enabled for the account, including whether the Pro models may be used (features.proModels). Free. Use it to check there is enough credit before scanning a batch.",
    "input_schema": {
      "type": "object",
      "properties": {},
      "additionalProperties": false
    }
  }
]

Reference implementation

A dependency-free Node module that implements both tools. scanReceipt uploads with an Idempotency-Key (retrying transient failures safely), polls at the pace Retry-After sets, and resolves with the result object plus the token and duplicate flag. For anything else it throws a TabscannerError carrying the HTTP status, the error code and a retryable flag.

To use it, register the tool definitions above with your agent framework. When the model calls scan_receipt, call scanReceipt(input) and return the JSON as the tool result; on a TabscannerError, return its message as a tool error. To serve the tools over the Model Context Protocol, wrap the same two functions as MCP tools.

The module is one of the tested samples; run it directly with an image path to see the output. Porting it to another language means taking that language's Quickstart sample and returning the result instead of printing it.

// Tabscanner as an agent tool. scanReceipt() uploads, polls and returns the result object,
// so the model sees one call instead of a polling loop. getAccount() returns the credit.
// Node 20 or newer, no packages.  Usage as a script: node agent-tool.js receipt.jpg [region] [model]
import { randomUUID } from 'node:crypto';
import { openAsBlob } from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

const API_URL = process.env.TABSCANNER_API_URL ?? 'https://api3.tabscanner.com';
const AUTH = { Authorization: `Bearer ${process.env.TABSCANNER_API_KEY}` };

const sleep = (seconds) => new Promise((resolve) => setTimeout(resolve, seconds * 1000));

/** An API error, carrying the problem-details code so the agent can explain it. */
export class TabscannerError extends Error {
  constructor(status, problem) {
    super(`${problem.code ?? status}: ${problem.detail ?? problem.title ?? 'request failed'}`);
    this.status = status;
    this.code = problem.code;
    this.retryable = status >= 500 || status === 409 || status === 429;
  }
}

async function request(route, init = {}) {
  const response = await fetch(`${API_URL}${route}`, { ...init, headers: { ...AUTH, ...init.headers } });
  const body = await response.json().catch(() => ({}));
  return { response, body };
}

/** Upload with retries: the same Idempotency-Key makes resending safe and never double-charges. */
async function upload(form, attempts = 4) {
  const headers = { 'Idempotency-Key': randomUUID() };
  for (let attempt = 1; ; attempt++) {
    try {
      const { response, body } = await request('/v2/receipts', { method: 'POST', headers, body: form });
      if (response.status === 202) return { token: body.token, duplicate: body.duplicate, retryAfter: Number(response.headers.get('retry-after') ?? 2) };
      const error = new TabscannerError(response.status, body);
      if (!error.retryable || attempt === attempts) throw error;
    } catch (error) {
      if (error instanceof TabscannerError && !error.retryable) throw error;
      if (attempt === attempts) throw error;
    }
    await sleep(2 ** (attempt - 1));
  }
}

/** Tool: scan_receipt. Resolves with the result object plus the token and duplicate flag. */
export async function scanReceipt({ image_path, region, document_type, model }) {
  const form = new FormData();
  const type = image_path.toLowerCase().endsWith('.png') ? 'image/png' : 'image/jpeg';
  form.append('file', await openAsBlob(image_path, { type }), path.basename(image_path));
  if (region) form.append('region', region);
  if (document_type) form.append('documentType', document_type);
  if (model) form.append('model', model);

  const { token, duplicate, retryAfter } = await upload(form);
  const deadline = Date.now() + (model ? 120_000 : 60_000);
  let wait = retryAfter;
  while (Date.now() < deadline) {
    await sleep(wait);
    const { response, body } = await request(`/v2/receipts/${token}`);
    if (response.status !== 200) throw new TabscannerError(response.status, body);
    if (body.status === 'done') return { token, duplicate, ...body.result };
    if (body.status === 'failed') throw new TabscannerError(422, { code: 'OCR_FAILED', detail: body.error.detail });
    wait = Number(response.headers.get('retry-after') ?? 2);
  }
  throw new Error(`Timed out waiting for the result. Token ${token} stays valid; fetch it later.`);
}

/** Tool: get_tabscanner_account. */
export async function getAccount() {
  const { response, body } = await request('/v2/account');
  if (response.status !== 200) throw new TabscannerError(response.status, body);
  return body;
}

// Run directly: node agent-tool.js receipt.jpg [region] [model]
if (process.argv[1] === fileURLToPath(import.meta.url)) {
  const [image_path = 'receipt.jpg', region, model] = process.argv.slice(2);
  const result = await scanReceipt({ image_path, region, model });
  console.log(`Establishment: ${result.establishment}`);
  console.log(`Total: ${result.total} ${result.currency}`);
  console.log(`Credit left: ${(await getAccount()).credit}`);
}

Which fields to trust

The result carries a confidence between 0 and 1 for every important value, and boolean shortcuts for the highest tier. An agent answering questions about a receipt should:

• Check total before relying on it: totalConfidence tells you how sure the reading is, and the arithmetic should agree. The lineItems amounts should sum to subTotal, and subTotal plus tip and service charges (and tax, when not already included) minus discounts should equal total. When confidence is low or the sums disagree, say so.
• Treat establishment as certain when validatedEstablishment is true; it was cross-checked against the merchant's phone number or address.
• Use dateISO for anything programmatic. It has no time zone; it is the local time printed on the receipt.
• Prefer descClean over desc for item names. A qty of 0 means no quantity was printed, not zero items.
• Read currency from the result rather than assuming it from region.
• Say so when a field is empty or has low confidence: the receipt did not print it, or it could not be read.

{
  "token": "dv9aLY1MVThnSJRo",
  "duplicate": false,
  "establishment": "GREENLEAF CAFE",
  "validatedEstablishment": false,
  "date": "2026-10-08 12:41:00",
  "dateISO": "2026-10-08T12:41:00",
  "total": 40.22,
  "totalConfidence": 0.99,
  "subTotal": 35.75,
  "serviceCharges": [4.47],
  "currency": "GBP",
  "lineItems": [
    { "qty": 2, "descClean": "Flat White", "lineTotal": 7.0 },
    { "qty": 1, "descClean": "Avocado Toast", "lineTotal": 9.5 }
  ]
}

Choosing a model

Standard OCR, the default, is the right first call for almost every receipt. A model reads the document more thoroughly at the cost of 10 to 20 seconds of processing. A sensible agent policy:

1. Scan without model.
2. If totalConfidence is low and the line items do not add up, or the user says the result is wrong, scan again with model: "harajuku". Tell the user this uses another credit.
3. If that result still doesn't add up, and features.proModels is true, try model: "harajuku-pro", the more thorough and slower version.

harajuku is available to every account. The -pro models need the Pro package: check features.proModels from get_tabscanner_account before offering one; if a scan fails with MODEL_NOT_PERMITTED, tell the user the Pro models are not included in their plan and fall back to harajuku. Nothing is charged for the refused request.

model	Profile	Use it for
omitted	Standard OCR	The default. Fastest turnaround, layout-aware OCR with regional rules.
harajuku	Model processing	Reads the document more thoroughly than standard OCR, for long, crumpled, faded or unusually laid out receipts and invoices.
harajuku-pro	Model processing, pro	A more thorough version of harajuku, for documents where accuracy matters most. Longer processing time.