copper3d / Modules / Utils/segmentation/tools/AiAssistTool / AiAssistTool

Class: AiAssistTool ​

Utils/segmentation/tools/AiAssistTool.AiAssistTool

Hierarchy ​

• BaseTool

  ↳ AiAssistTool

Table of contents ​

Constructors ​

• constructor

Properties ​

• activeLabel
• activeSlice
• box
• committedVol
• ctx
• dragScreen
• dragScreenStart
• dragStart
• dragging
• hidePointMarkers
• host
• hoverScreen
• lasso
• lassoHoverIdx
• lassoRedo
• lassoUndo
• lassoVerts
• lastLassoDownPos
• lastLassoDownTime
• onLassoChange
• onPrompt
• points
• polarity
• promptTool
• scratchVol
• screenScribble
• scribble
• scribbleSize
• snapshotData
• LASSO_HIT_R

Methods ​

• applyMask
• applySliceRle
• cancelLasso
• clearLassoEditing
• clearSegment
• commitRegion
• discard
• drawSeedMarker
• emit
• enter
• exit
• finishLasso
• getActiveLabel
• getLassoVertCount
• getScratchSegments
• getScratchSlices
• getScratchVolume
• isLassoEditing
• lassoRedoAction
• lassoUndoAction
• lassoVertAt
• newSegment
• notifyLasso
• onLassoPointerDown
• onPointerDown
• onPointerMove
• onPointerUp
• orderedLassoVerts
• pushLassoUndo
• renderLassoOverlay
• renderOverlay
• resetPrompts
• sampleClosedSpline
• setActiveSegment
• setContext
• setPolarity
• setPromptTool
• setScribbleSize
• setSegmentColor
• sliceDims
• syncSlice
• toVoxel
• voxelToScreen

Constructors ​

constructor ​

• new AiAssistTool(ctx, host)

Parameters ​

Overrides ​

BaseTool.constructor

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:137

Properties ​

activeLabel ​

• Private activeLabel: number = 1

The label value (1-255) the AI currently paints into = the active Segmentation. Each Segmentation is a label value in the single-channel scratch volume; its colour lives in the volume's colorMap (set via setSegmentColor).

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:91

activeSlice ​

• Private activeSlice: number = -1

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:101

box ​

• Private box: undefined | { label: number ; x0: number ; x1: number ; y0: number ; y1: number }

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:100

committedVol ​

• Private committedVol: null | MaskVolume = null

Frozen regions ("New region" commits here). Pixels recorded here survive a later re-prediction — even on the SAME channel — so multiple separate regions can be drawn without the newest prediction wiping the previous ones. Null = nothing frozen yet (applySliceRle then behaves as a single live region).

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:84

ctx ​

• Protected ctx: ToolContext

Inherited from ​

BaseTool.ctx

Defined in ​

src/Utils/segmentation/tools/BaseTool.ts:37

dragScreen ​

• Private dragScreen: null | { x: number ; y: number } = null

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:122

dragScreenStart ​

• Private dragScreenStart: null | { x: number ; y: number } = null

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:121

dragStart ​

• Private dragStart: null | { x: number ; y: number } = null

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:119

dragging ​

• Private dragging: boolean = false

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:118

hidePointMarkers ​

• Private hidePointMarkers: boolean = false

Hide the point seed markers once a prediction has painted (only the mask should remain). A new point click re-shows them until its mask returns.

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:129

host ​

• Private host: AiAssistHostDeps

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:74

hoverScreen ​

• Private hoverScreen: null | { x: number ; y: number } = null

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:126

lasso ​

• Private lasso: AiPromptPoint[] = []

Lasso contour sent to the backend (densified along the curve at finish).

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:99

lassoHoverIdx ​

• Private lassoHoverIdx: number = -1

Index of the vertex currently hovered (shows a red ✕; click removes it). −1 = none.

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:110

lassoRedo ​

• Private lassoRedo: LassoVert[][] = []

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:108

lassoUndo ​

• Private lassoUndo: LassoVert[][] = []

Undo / redo snapshot stacks of the vertex list (for Ctrl+Z / Ctrl+Y).

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:107

lassoVerts ​

• Private lassoVerts: LassoVert[] = []

Placed lasso vertices (voxel + screen + polarity). Empty = not editing.

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:105

lastLassoDownPos ​

• Private lastLassoDownPos: Object

Type declaration ​

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:115

lastLassoDownTime ​

• Private lastLassoDownTime: number = 0

Double-click detection (a dbl-click finishes the lasso).

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:114

onLassoChange ​

• onLassoChange: null | (count: number, editing: boolean) => void = null

Fired whenever the lasso vertex set changes (add/delete/undo/redo/finish/cancel), so the panel can reactively show the Finish button + vertex count.

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:135

onPrompt ​

• onPrompt: null | (payload: AiPromptPayload) => void = null

Fired when a prompt gesture completes — app calls backend, then applyMask().

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:132

points ​

• Private points: AiPromptPoint[] = []

Accumulated prompts for the CURRENT slice; reset on slice/axis change.

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:96

polarity ​

• Private polarity: number = 1

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:87

promptTool ​

• Private promptTool: AiPromptTool = "point"

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:86

scratchVol ​

• Private scratchVol: null | MaskVolume = null

Independent scratch volume (also registered at maskData.volumes['aiScratch']).

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:77

screenScribble ​

• Private screenScribble: { x: number ; y: number }[] = []

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:123

scribble ​

• Private scribble: AiPromptPoint[] = []

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:97

scribbleSize ​

• Private scribbleSize: number = 5

Scribble brush radius (px), user-adjustable via slider.

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:93

snapshotData ​

• Private snapshotData: null | Uint8Array = null

Snapshot of the scratch buffer taken on enter (for discard).

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:79

LASSO_HIT_R ​

▪ Static Private Readonly LASSO_HIT_R: 8

Screen-pixel radius for hit-testing a hovered vertex.

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:112

Methods ​

applyMask ​

▸ applyMask(result): void

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:668

applySliceRle ​

▸ Private applySliceRle(rle, axis, sliceIndex, width, height): void

Merge one RLE-encoded predicted slice into the scratch volume at sliceIndex. Predicted pixels → current channel; stale current-channel pixels cleared UNLESS they belong to a frozen (committed) region; OTHER channels preserved.

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:687

cancelLasso ​

▸ cancelLasso(): void

Public: abandon the in-progress lasso (Esc).

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:602

clearLassoEditing ​

▸ Private clearLassoEditing(): void

Drop the in-progress lasso vertices + undo/redo history (does not touch a prediction already painted from a finished lasso).

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:354

clearSegment ​

▸ clearSegment(label): void

Delete a Segmentation: erase every voxel painted with its label from BOTH the live scratch and the frozen (committed) volume, so the region disappears from the view (otherwise orphaned label voxels would render with the wrong colour). Goes via setRawData so the volume's version counter bumps (overlay repaints).

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:185

commitRegion ​

▸ commitRegion(): void

"New region": freeze every voxel painted so far so it survives later re-predictions (even on the same channel), then start a fresh prompt set. This is what makes drawing multiple separate regions actually work — without it the next prediction's same-channel cleanup erases the previous region.

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:325

discard ​

▸ discard(): void

Discard everything painted since enter() — restore the snapshot.

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:313

drawSeedMarker ​

▸ Private drawSeedMarker(ctx, x, y, fg): void

A seed marker (small filled dot + ring) for point prompts.

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:815

emit ​

▸ Private emit(): void

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:544

enter ​

▸ enter(): void

Create (or reuse) the scratch volume sized to the layer grid and register it under maskData.volumes['aiScratch']. Snapshots the empty buffer.

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:275

exit ​

▸ exit(): void

Remove the scratch volume from the registry and drop references.

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:301

finishLasso ​

▸ finishLasso(): void

Public: close the curve, densify it, and send it to the backend as the lasso contour. Needs ≥3 vertices (a real area). Then clears the editing state.

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:627

getActiveLabel ​

▸ getActiveLabel(): number

Returns ​

number

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:166

getLassoVertCount ​

▸ getLassoVertCount(): number

Public: number of placed lasso vertices (drives the Finish button label/enable).

Returns ​

number

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:607

getScratchSegments ​

▸ getScratchSegments(): null | { axis: "z" ; height: number ; segments: { label: number ; slices: { rle: number[] ; sliceIndex: number }[] }[] ; width: number }

Per-SEGMENTATION serialization for the 3D build: one binary RLE per non-empty z-slice PER label (no binarize-merge), so the backend writes a multi-label NIfTI and colours the GLB per segmentation. Includes both live and frozen regions (both live in scratchVol).

Returns ​

null | { axis: "z" ; height: number ; segments: { label: number ; slices: { rle: number[] ; sliceIndex: number }[] }[] ; width: number }

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:239

getScratchSlices ​

▸ getScratchSlices(): null | { axis: "z" ; height: number ; slices: { rle: number[] ; sliceIndex: number }[] ; width: number }

Serialize the scratch volume as per-z-slice RLE (only NON-empty slices), for persisting to ai_generated_nii_LPS on the backend. Binary (any channel → 1); RLE is alternating run lengths starting with a 0-run — the exact format the backend's rle_decode expects. Uses getSliceUint8 (the same path the clinician layer's /api/mask/replace uses) so orientation matches.

Returns ​

null | { axis: "z" ; height: number ; slices: { rle: number[] ; sliceIndex: number }[] ; width: number }

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:206

getScratchVolume ​

▸ getScratchVolume(): null | MaskVolume

Returns ​

null | MaskVolume

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:199

isLassoEditing ​

▸ isLassoEditing(): boolean

Public: true while the user is placing lasso vertices (≥1 vertex).

Returns ​

boolean

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:605

lassoRedoAction ​

▸ lassoRedoAction(): void

Public: redo (Ctrl+Y / Ctrl+Shift+Z while editing).

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:593

lassoUndoAction ​

▸ lassoUndoAction(): void

Public: undo the last add/delete of a lasso vertex (Ctrl+Z while editing).

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:584

lassoVertAt ​

▸ Private lassoVertAt(sx, sy): number

Index of the lasso vertex under (sx,sy) within the hit radius, else −1. Uses live voxel→screen so it tracks pan/zoom. Topmost (latest) wins.

Parameters ​

Returns ​

number

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:565

newSegment ​

▸ newSegment(label): void

"New segmentation": freeze the current regions (so they persist) and switch to a fresh label, exactly like the old "New region" but targeting a new label.

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:176

notifyLasso ​

▸ Private notifyLasso(): void

Tell the app layer the lasso vertex set changed (drives the Finish button).

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:363

onLassoPointerDown ​

▸ Private onLassoPointerDown(e, vx, vy): void

Lasso click: double-click finishes; click on a vertex deletes it; otherwise add a vertex. Nothing is sent to the backend until finishLasso().

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:469

onPointerDown ​

▸ onPointerDown(e): void

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:438

onPointerMove ​

▸ onPointerMove(e): void

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:500

onPointerUp ​

▸ onPointerUp(_e): void

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:525

orderedLassoVerts ​

▸ Private orderedLassoVerts(): LassoVert[]

Vertices ordered by polar angle around their centroid, so the closed curve is a SIMPLE (non-self-intersecting) loop regardless of click order — otherwise connecting points in click order + closing easily produces a bow-tie crossing. Trade-off: deep concavities collapse to their star-shaped hull, which is fine for a "rough loop around a blob".

Returns ​

LassoVert[]

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:614

pushLassoUndo ​

▸ Private pushLassoUndo(): void

Snapshot the vertex list for undo (clears redo).

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:577

renderLassoOverlay ​

▸ Private renderLassoOverlay(ctx): void

Render the lasso editing overlay (curve + fill + vertices + hover ✕).

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:829

renderOverlay ​

▸ renderOverlay(targetCtx): void

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:725

resetPrompts ​

▸ resetPrompts(): void

Clear the in-progress prompt set (e.g. slice/axis change). Does NOT freeze — use commitRegion() for the "New region" action.

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:341

sampleClosedSpline ​

▸ Private sampleClosedSpline(pts, segs): { x: number ; y: number }[]

Sample a smooth CLOSED Catmull-Rom spline through pts → denser polyline. segs samples per control-point span. ❤️ pts → returns a copy unchanged.

Parameters ​

Returns ​

{ x: number ; y: number }[]

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:642

setActiveSegment ​

▸ setActiveSegment(label): void

Select the active Segmentation by its label value (1-255). Switching starts a fresh prompt set — otherwise accumulated prompts would be re-predicted and repainted into the new label, recolouring the previous segmentation's region.

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:161

setContext ​

▸ setContext(ctx): void

Parameters ​

Returns ​

void

Inherited from ​

BaseTool.setContext

Defined in ​

src/Utils/segmentation/tools/BaseTool.ts:43

setPolarity ​

▸ setPolarity(label): void

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:154

setPromptTool ​

▸ setPromptTool(tool): void

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:144

setScribbleSize ​

▸ setScribbleSize(size): void

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:155

setSegmentColor ​

▸ setSegmentColor(label, color): void

Set a Segmentation's colour (its label's entry in the scratch volume colorMap). The 2D overlay repaints next frame (copper3d's render loop re-reads the map).

Parameters ​

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:170

sliceDims ​

▸ Private sliceDims(): null | { h: number ; w: number }

Voxel-slice (width,height) for the current axis.

Returns ​

null | { h: number ; w: number }

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:380

syncSlice ​

▸ Private syncSlice(): void

Reset the accumulated prompt set when the user scrubs to a new slice.

Returns ​

void

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:536

toVoxel ​

▸ Private toVoxel(e): null | { h: number ; vx: number ; vy: number ; w: number }

Screen offset → voxel-slice (vx,vy) for the current axis.

Parameters ​

Returns ​

null | { h: number ; vx: number ; vy: number ; w: number }

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:392

voxelToScreen ​

▸ Private voxelToScreen(vx, vy): null | { x: number ; y: number }

Voxel-slice (vx,vy) → screen offset (px). Exact inverse of toVoxel, so markers / lasso curves drawn from voxel coords track pan AND zoom each frame (the freehand previews store screen px directly; persistent overlays don't).

Parameters ​

Returns ​

null | { x: number ; y: number }

Defined in ​

src/Utils/segmentation/tools/AiAssistTool.ts:418