Prompt packs

A prompt pack holds every instruction the VLM receives: how to classify an item, how to verify a region, and how to answer. The text is data, not code. Packs are editable per project, versioned, and can be rolled back.

API=http://localhost:4603 and project cars are assumed.

Concepts

• Fields. A pack has 14 text fields, four more for attributes and two maps. Fields come in system and user pairs, one pair per VLM call:

  Call	Fields
  classify	class_system, class_user_template
  open_classify	open_class_system, open_class_user_template
  combined	combined_system, combined_user_template
  combined_batch	combined_batch_system, combined_batch_rules
  region_verify	region_system, region_user
  region_verify_batch	region_batch_system, region_batch_user
  region_visible	region_visible_system, region_visible_user

  The maps are class_descriptions (class name to a short description for the model) and synonyms (a label the model may answer, mapped to the registry class name it stands for; matching tries the space and underscore forms).

  An optional list, proposal_denylist, holds case-insensitive glob patterns (blurry_*, *_scene) for proposed new-class names that are scene or image-quality words rather than classes. A matching proposal is dropped, so the crop counts as "nothing fits" and does not reach the new-class queue. It defaults to empty for a stored pack that omits it; the shipped generic packs carry a starter list covering blur, quality, empty and scene words plus abstract*, *_background and *_object.

  An optional integer, detector_hint_min_confidence_pct (0 = off, the default; at most 100), adds one line per image naming the item's stored detector class when the detector confidence is at least that percent. It is a hint, not a constraint. On a public COCO oracle (539 crops) it raised overall accuracy from 0.775 to 0.853 (0.7846 to 0.8478 with ten validated classes seeded). The detector is COCO-trained and the oracle is COCO ground truth, so the gain may be optimistic for other domains. A registry-prior variant was measured, gave no gain and was removed.

• Attributes. A pack may declare up to 16 attributes: free-form facts a VLM reports about an item or a box, such as a car's make or a wheel's rim material. Each entry has a key (^[a-z][a-z0-9_]{0,31}$, unique), a description (up to 300 characters), an optional closed vocabulary values (up to 50 lowercase values of up to 48 characters) and applies_to (item, the default, or box). Two more calls report them, and their fields stay empty while the pack declares no attributes:

  Call	Fields	Used for
  assist	assist_system, assist_user_template	Classify and report item attributes in one call
  attributes	attributes_system, attributes_user_template	Attributes alone: items without classify, and box sub-crops

  assist_user_template takes {class_list} and {attribute_block} and may use {n_images}; attributes_user_template takes {attribute_block} and may use {n_images}. The server renders {attribute_block} from attributes (key, description, allowed values), so the author does not format it. A reply entry carries attributes as {key: {"value": ..., "confidence": "high"|"medium"|"low"}}. The parser drops keys the pack did not declare, normalises values (trimmed, lowercase, 48 characters), drops a value outside a closed vocabulary and counts it, and reads a missing confidence as low. An attribute is a suggestion: it never changes a class or a box. Item attributes need all four prompt fields filled, box attributes the attributes_* pair. examples/prompt_packs/vehicle_wheel.json declares make, model and body_style for the car and rim_material for the wheel box. The region worker's combined call also asks for the item attributes on the items it classifies, and a whole-project auto-label run does with vlm_attributes=true; both store them in the item's assist slot. Preview them on up to 64 crops with POST .../prompt_packs/test and call: "assist" or "attributes".

• Placeholders. Only a few fields are formatted with str.format: class_user_template and open_class_user_template accept class_names_csv; combined_user_template accepts class_block and region_block; the two attribute templates are described above. Each formatted field must use all of its required placeholders and no others. A literal brace in those fields is written doubled. Every other field is sent verbatim, and a placeholder in it is an error.

• Reply keys. The worker parses the model's JSON reply. Each call has required keys that the pack text must name, for example is_region and confidence for region_verify. GET .../prompt_packs/schema lists them per field.

• Sources. builtin (the shipped generic packs), file (loaded from OP_PROMPT_PACK_PATH or OP_PROMPT_PACK_PATHS), template (shipped under examples/prompt_packs/) and stored (saved in the project). Only stored packs are editable. The others are read-only; clone them to edit.

• Revisions. Every save of a stored pack is a new numbered revision. Old revisions stay readable and can be activated again.

• Activation. One pack is active per project. Activation passes a gate that pairs the pack with the active region profile and VLM. A change to the pack takes effect only when it is activated.

Routes

Call	Purpose
GET /curation/projects/{project}/prompt_packs	List packs, templates and the active one
POST /curation/projects/{project}/prompt_packs	Create a stored pack
GET /curation/projects/{project}/prompt_packs/schema	Field, call and placeholder specs
POST /curation/projects/{project}/prompt_packs/validate	Validate a draft; never writes
POST /curation/projects/{project}/prompt_packs/test	Dry run against crops, see Test on a crop
GET /curation/projects/{project}/prompt_packs/active	The active pack and what workers applied
POST /curation/projects/{project}/prompt_packs/active/rollback	Re-activate the previous pack
GET /curation/projects/{project}/prompt_packs/{name}	One pack
PUT /curation/projects/{project}/prompt_packs/{name}	Save a new revision
DELETE /curation/projects/{project}/prompt_packs/{name}	Delete a stored pack
POST /curation/projects/{project}/prompt_packs/{name}/activate	Activate
POST /curation/projects/{project}/prompt_packs/{name}/clone	Copy into a new stored pack
GET /curation/projects/{project}/prompt_packs/{name}/revisions	Revision history
GET /curation/projects/{project}/prompt_packs/{name}/revisions/{revision}	One past revision

List and inspect

curl -s "$API/curation/projects/cars/prompt_packs"
curl -s "$API/curation/projects/cars/prompt_packs/schema"
curl -s "$API/curation/projects/cars/prompt_packs/vehicle_wheel"

The list response has packs (each with name, source, read_only, revision, active, active_revision, asks_region_text, etag), templates, active ({name, revision}) and config_revision. A pack document has body (the fields), description, revision, cloned_from, created_at, updated_at, etag and, after a write, validation.

Validate

curl -s -X POST "$API/curation/projects/cars/prompt_packs/validate?profile=vehicle_wheel" \
  -H 'Content-Type: application/json' \
  -d '{"name": "cars_v1", "body": {"region_system": "You verify a wheel.", "region_user": "Reply as JSON with is_region and confidence."}}'

The body may be partial. Missing content is reported as issues, not as an HTTP error. The optional profile query parameter pairs the pack with a named region profile (default: the active one). The response is {ok, errors, warnings, force_allowed}. Each issue has code, severity, field, message and detail.

Code	Severity	Meaning
pack_name_invalid, pack_name_reserved	error	Name must match ^[a-z0-9][a-z0-9_.-]{1,63}$; off, none and default are reserved
name_conflict	error	Name already used
pack_field_missing, pack_field_empty	error	A text field is absent or blank; proposal_denylist is not a list of strings, or holds a blank pattern; detector_hint_min_confidence_pct is not an integer
pack_field_too_long	error	A field over 20000 characters, more than 16 attributes, a map or proposal_denylist over 500 entries, or a denylist pattern over 200 characters
pack_field_out_of_range	error	detector_hint_min_confidence_pct is outside 0 to 100
pack_template_format_error	error	The field cannot be formatted (for example a single brace)
pack_placeholder_missing, pack_placeholder_unknown	error	A formatted field lacks a required placeholder or uses another
pack_placeholder_in_verbatim_field	error	A placeholder in a field that is not formatted
pack_reply_key_missing	error	The text does not name a reply key the call needs
pack_attribute_invalid	error	An attributes entry has a bad key, description, values or applies_to, or attributes is not a list
pack_attribute_duplicate	error	Two attributes share a key
pack_multi_region_keys_missing	warning on validate, error on activation	The profile allows more than one box per item and the combined prompts do not name the per-box keys
pack_asks_text_profile_text_free	warning	The pack asks for region text and the profile reads none
pack_no_text_profile_reads_text	warning	The profile reads text through the VLM and the pack never asks
pack_synonym_target_unknown, pack_description_class_unknown	warning	A map names a class that is not in the registry
pack_example_values	info	Quoted example values the model may echo back; they are rejected as text

Create, edit, clone

The simplest start is to clone a template or built-in pack:

curl -s -X POST "$API/curation/projects/cars/prompt_packs/vehicle_wheel/clone" \
  -H 'Content-Type: application/json' \
  -d '{"new_name": "cars_v1", "description": "My wheel prompts", "source": "template"}'

source is one of builtin, file, template, stored and picks which copy of the name to clone when the name exists in several places. revision picks a stored revision. from_project clones a stored pack from another project. The answer is 201 with the new pack document.

Create from scratch with every text field set (see examples/prompt_packs/vehicle_wheel.json for a full body):

curl -s -X POST "$API/curation/projects/cars/prompt_packs" \
  -H 'Content-Type: application/json' \
  -d '{"name": "cars_v1", "description": "", "body": { ...all fields... }}'

Save a new revision of a stored pack. expected_revision is the revision you read:

curl -s -X PUT "$API/curation/projects/cars/prompt_packs/cars_v1" \
  -H 'Content-Type: application/json' \
  -d '{"expected_revision": 1, "description": "tighter wording", "body": { ...all fields... }}'

Both calls refuse a pack that has validation errors with 422. Warnings are returned in validation and do not block.

Activate, roll back, delete

curl -s -X POST "$API/curation/projects/cars/prompt_packs/cars_v1/activate" \
  -H 'Content-Type: application/json' \
  -d '{"revision": null, "force": false, "expected_active": null}'

curl -s -X POST "$API/curation/projects/cars/prompt_packs/active/rollback" \
  -H 'Content-Type: application/json' -d '{"expected_active": null}'

curl -s -X DELETE "$API/curation/projects/cars/prompt_packs/cars_v1?expected_revision=2"

revision null activates the latest. expected_active (an object with name and revision) makes the call fail with 409 if another writer activated first. force does not bypass any pack error: pack validation has no bypassable codes. Activation re-runs the pairing checks with the active region profile and VLM and answers 422 validation_failed with the report when they fail. The response has active, previous, source, activated_at, applied (per worker: which pack revision it runs, and lagging) and validation.

Rollback re-activates the previous pack through the same gate. Delete answers 204.

Revisions

curl -s "$API/curation/projects/cars/prompt_packs/cars_v1/revisions"
curl -s "$API/curation/projects/cars/prompt_packs/cars_v1/revisions/1"

Loading packs from files

A deployment can ship packs as JSON files. OP_PROMPT_PACK_PATH names the default pack file, and OP_PROMPT_PACK_PATHS lists more, comma separated. Both appear with source file and are read-only. The generic built-in packs are always listed.

Errors

Status	error	When
404	not_found	Unknown pack, or no pack of that source
404	unknown_revision	The pack has no such revision
403	read_only	Save or delete of a built-in, file or template pack
409	name_conflict	Create or clone onto a used name
409	revision_conflict	expected_revision is stale; current_revision is returned
409	in_use	Delete of the active pack
409	active_conflict	Another writer changed the activation; current is returned
409	no_previous, previous_deleted	Nothing to roll back to
422	validation_failed	Validation errors; report lists them

Cropwright

The prompt pack editor with reply-key hints and the validation summary (a built-in, read-only pack) (captured from Cropwright against the public COCO sample projects)

See also

• Selective VLM and segmenter assist, where attributes are used
• Test on a crop
• Region profiles
• VLM selection
• Multi-box regions
• Wheel example