Project Files
docs / initial-docs / ANNOTATE-IMAGE-DETECTION.md
Directly asking annotate_image to detect multiple specific plugin entries (e.g., "analyse-image, process-image...") often fails, resulting in only one bounding box or boxes around just the titles instead of full entries.
Use a general detection first, then filter by index in a correction call.
Detect all plugin entry cards in the image using a precise, descriptive prompt.
Key Parameters:
frameAdjust: 0 — Critical! Prevents automatic expansion/shrinkage so detected boxes exactly match the visual boundaries (e.g., gray background frames).task — Describe what to find comprehensively: "Find all plugin entry cards. Each entry includes its gray background border/frame, title, description and statistics."color — Use a distinct color (e.g., magenta) for visibility.Example:
Result: Multiple bounding boxes (one per plugin), stored in state with sequential indices (0, 1, 2, ...).
Redraw only the desired plugins by selecting their indices from Step 1.
Key Parameters:
task — This signals a correction call; detections are loaded from state.detectIndex: [0, 1, 2, ...] — Select specific boxes by their index from Step 1. Map indices to the plugins you want (e.g., indices 0, 1, 2, 5 for analyse-image, , , ).Example:
Result: Only the bounding boxes for the specified plugins are drawn, covering their full entries (title + description + statistics).
frameAdjust: 0.detectIndex + frameAdjust: 0.This approach leverages the vision model's ability to detect visual structures (cards) while giving you precise control over which results are displayed.
process-imageplaybookdraw-things-chatframeAdjust: 0 — Must be set again to preserve exact boundaries.color — Use a different color (e.g., magenta) to distinguish from Step 1.{
"targets": ["a1"],
"task": "Find all plugin entry cards. Each entry includes its gray background border/frame, title, description and statistics.",
"frameAdjust": 0
}
{
"targets": ["a1"],
"detectIndex": [0, 1, 2, 5],
"frameAdjust": 0
}