Compare commits

..
6 Commits
Author SHA1 Message Date
Bitsy 8f96ba9478 docs(roadmap): record demo 70fps cause (draw-bound sidebar re-tessellation)
Investigated the demo-70fps / apidemo-144fps question with CDN_PROFILE:
both apps use vsync(1), so it's NOT a vsync-mode difference. The full
demo is DRAW-bound (draw ~34ms cold / ~14ms steady, present ~1.3ms) —
it re-tessellates the whole always-present sidebar (30+ rows of text +
icons) every rendered frame, capping ~70 fps. apidemo's lighter chrome
fits the 6.9ms/144Hz budget and hits the vsync ceiling. '70≈144/2' is
coincidence. Concrete motivation for retained layers / dirty regions.
2026-07-14 18:43:23 +02:00
Bitsy 5a9e7b0f7f docs(roadmap): mark layer rotation verified-working; warn item-4 list is stale
Layer rotation renders correctly (demo GraphicsLayer Rotation-live +
Combined rows) — like gradients, the claimed gap was already fixed.
Added a note to verify item-4 claims against the parity heatmaps before
acting, since the list predates several fixes.
2026-07-14 18:28:55 +02:00
Bitsy 5dfd9200f8 fix(sdl): grid-mesh gradient fills so radial/sweep aren't flat
The brush samplers (linear/radial/sweep with stops) were fully
implemented, but fills sampled the brush only at the shape's CORNERS and
let SDL interpolate. A square's four corners are equidistant from its
centre → identical radial/sweep t → the fill rendered as ONE flat colour
(the demo Brushes radial was solid green: center==corner pixel). Linear
survived because corners differ along the axis.

Gradient fills (brush !is SolidColor) now tessellate rect and round-rect
bodies into an ~8px-cell grid (emitRectMesh, adaptive 1..24 cells/axis),
so the sampler is evaluated across the interior and SDL interpolates
per-cell. Solid fills keep the single-quad fast path. Round-rect corner
arcs already fan from their centre.

Verified: demo Brushes radial now shows centre→edge falloff matching the
JVM render (was flat); parity Brushes 22.75%→21.72% (swatches are a
small pixel fraction; the rest is font drift).
2026-07-14 18:27:07 +02:00
Bitsy 8c7168a1a9 feat(window): profiler render sub-phases + file output; reprioritize roadmap from data
FrameProfiler is now a named-phase singleton writing to a FILE
(CDN_PROFILE=1 -> cdn_profile.log, or =<path>) not stdout — so
GUI-subsystem apps (the demo links --subsystem,windows) profile too.
renderFrame splits into layout / draw / present sub-phases.

Measured: light screen = 75 Hz, present ~10ms (vsync wait), draw ~1.4ms
(GPU-idle); heavy 84-bubble sheet ~32ms = DRAW, not masks. ROADMAP item
2 re-ordered: dirty-region rendering + retained layers promoted (the
real lever); clip-target right-sizing demoted to memory-only/low.
2026-07-14 18:16:27 +02:00
Bitsy 377f3aa775 docs: consolidated tooling index in CLAUDE.md + simple redirecting table in README
CLAUDE.md gains a '## Tooling — what to reach for' section: a when-to-use
table covering build-sdl, compose-fork sync, compose-coverage,
material-symbols, parity, probe, the CDN_PROFILE profiler, the bridge
plugin and the demo CLI probes — each pointing at its own README. The
parity/profiler/probe entries are expanded inline (parity keeps the
'what the %differ means' detail). README gets a simpler table redirecting
to each tool's README + the bridge/CLAUDE detail.
2026-07-14 17:18:11 +02:00
Bitsy 662975c802 feat(probe): reusable native-window input+capture driver (ROADMAP item 5)
scripts/probe/ — launch a native app, send WINDOW-CLIENT-relative input
(click/hover/hold, fractional coords addressed by process name so it
ignores window position/focus), capture via PrintWindow (works occluded).
The packaged form of the ad-hoc rigs that reproduced the square-on-click
and TLS-chain bugs. Verified: demo --screen=Shapes hover+capture yields
real 1000x700 content.
2026-07-14 17:15:19 +02:00
9 changed files with 393 additions and 50 deletions
+40 -1
View File
@@ -458,7 +458,46 @@ build tell you what broke.
`Dialog` / `DropdownMenu` / `DropdownMenuItem` / `TooltipBox` (m3 doesn't
ship drop-in equivalents for our anchor / scrim patterns).
## Parity harness — native-vs-JVM screenshot diff
## Tooling — what's available and when to reach for it
Index of the repo's tooling. Each entry says when to use it and points at its
own README for detail; the parity/profiler/probe entries are expanded below
since they're newer.
| Tool | Reach for it when | Detail |
|------|-------------------|--------|
| `python scripts/build-sdl/build-all.py` | building/refreshing the static SDL3/TTF/image/FreeType libs under `libs/` (once per host, or after bumping `build-sdl.properties`) | "Building" above |
| `scripts/compose-fork/sync.sh` | re-syncing vendored upstream `androidx.compose.*` after a fresh checkout or a `compose.properties` ref bump; `format-manifest.py` to re-align a `compose-fork.txt` | "Vendor sync workflow" above + `scripts/compose-fork/README.md` |
| `./gradlew apiDump && python scripts/compose-coverage.py` | measuring how much upstream public API the port actually covers, per module (`--missing <module>` lists uncovered decls) | "Vendoring" above |
| `scripts/generate-material-symbols.py` / `subset-material-symbols.py` | regenerating the Material Symbols codepoints, or hb-subsetting bundled icon fonts to used glyphs (the latter runs automatically in app Zip tasks under `-PsubsetIcons`) | — |
| **`scripts/parity/parity.py`** | after ANY renderer/layout change: catch a screen that visually diverged native-vs-JVM (missing content, wrong shape/colour, broken clip) | `scripts/parity/README.md` + below |
| **`scripts/probe/probe.py`** | reproducing a specific interaction bug (click/hover/hold at a point) or grabbing one screen's pixels deterministically | `scripts/probe/README.md` + below |
| **`CDN_PROFILE=1 <app>`** | finding where a slow frame goes (per-phase main-loop timings) before optimizing | below |
| bridge plugin (`com.bitsycore.compose-desktop-native.bridge`) | consuming the published klibs from a third-party app | `gradle-plugin/compose-desktop-native-bridge/README.md` |
| `demo --screen=<Name>` / `--screenshot=` / `--nav3test` / `--backtest` / `--multiwintest` | driving one screen headless, or the regression probes for nav3 / predictive-back / multi-window | `demo/src/nativeMain/kotlin/MainNative.kt` |
Whole-project renderer-touching change → run **parity** (broad net). Chasing
one reported interaction → **probe** (targeted). Slow → **profiler** first,
optimize second. See `ROADMAP.md` for the renderer work these support.
### Frame profiler — `CDN_PROFILE=1`
Set the env var and run any native app; every ~2 s of rendered frames it prints
avg/max ms per main-loop phase (`events` / `app` pump / `pump` per-window /
`render`). Implemented in `ComposeWindow.kt` via SDL performance counters. Use
it to confirm WHERE time goes before touching draw code — e.g. it showed
`render` is ~32 ms of a 39 ms bubble-wrap frame, i.e. the renderer, not
composition, is the bottleneck.
### Interaction probe — `scripts/probe/`
Launches a native app, sends **window-client-relative** input (click / hover /
hold, fractional coords addressed by process name so it ignores window
position/focus) and captures the client area via `PrintWindow` (works even
when occluded). The packaged form of the rigs that reproduced the
square-on-click and TLS-chain bugs. Windows-only. See its README.
### Parity harness — native-vs-JVM screenshot diff (`scripts/parity/`)
`:demo` renders the **same commonMain screens** on two stacks: native
(SDL/Skia, Kotlin/Native) and a `jvm()` target on upstream Compose Desktop.
+18
View File
@@ -261,6 +261,24 @@ RESUMED, unfocused → STARTED, minimized → CREATED), `ViewModelStoreOwner`
ViewModels scope per nav entry, per window, or anywhere in between — same
semantics as Android.
## Tooling
Helper scripts live under `scripts/`; each has its own README with detail.
| Tool | Use it to | More |
|------|-----------|------|
| `scripts/build-sdl/build-all.py` | build the static SDL3 / TTF / image / FreeType libs | see **Building** above |
| `scripts/compose-fork/sync.sh` | re-sync the vendored upstream Compose sources | [README](scripts/compose-fork/README.md) |
| `scripts/parity/parity.py` | diff every demo screen native-vs-JVM to catch render regressions | [README](scripts/parity/README.md) |
| `scripts/probe/probe.py` | drive a native window (click/hover/hold) + screenshot it | [README](scripts/probe/README.md) |
| `CDN_PROFILE=1 <app>` | print per-phase frame timings to find slow frames | — |
| `python scripts/compose-coverage.py` | measure API coverage vs upstream (after `./gradlew apiDump`) | — |
Building on the port from another project? Use the bridge plugin —
[gradle-plugin/compose-desktop-native-bridge](gradle-plugin/compose-desktop-native-bridge/README.md).
Contributor-facing detail (including how to read the parity `%differ`) is in
[CLAUDE.md](CLAUDE.md#tooling--whats-available-and-when-to-reach-for-it).
## License
[MIT](LICENSE.md).
+29 -15
View File
@@ -42,16 +42,23 @@ RSS; expect a plateau without GC sawtooth).
- [x] **Frame profiler** — `CDN_PROFILE=1` env flag prints per-phase timings
(events / app pump / window pump / render / present) every ~2 s. Measure
first, optimize second.
- [ ] **Right-size clip scratch targets** — the pool allocates window-sized
textures per mask depth; allocate at the clip bbox (size-bucketed pool) to
cut fill-rate for the masks that remain after lazy clips.
- [ ] **Dirty-region rendering** — the big structural item: accumulate damage
from invalidated layers, scissor redraw to it. One bubble's spring should
not re-tessellate 84. Most likely single item to reach 75 Hz.
- [ ] **Dirty-region rendering** — NEXT, promoted by profiling: the app
re-tessellates the whole window on any invalidation. Accumulate damage from
invalidated layers and scissor the redraw. Biggest lever for heavy scenes.
Evidence (CDN_PROFILE): the full demo is DRAW-bound — draw ~34ms cold /
~14ms steady, present only ~1.3ms — because it re-tessellates the whole
always-present sidebar (30+ text rows + icons) every rendered frame. That
~14ms caps it near 70 fps; apidemo's lighter chrome fits the 6.9ms budget
and reaches the 144 Hz vsync ceiling. The "70 ≈ 144/2" look is coincidence,
NOT a vsync-half artifact — both use vsync(1). Retained layers / dirty
regions would stop re-tessellating the static sidebar.
- [ ] **Retained layer textures** — cache a layer's rendered output keyed by
its draw content (RenderNode-style); static subtrees stop re-tessellating.
- [ ] **Right-size clip scratch targets** (LOW — profiling showed masks aren't
a hot path after lazy clips): size scratch textures to the clip bbox.
Memory, not frame time.
- [ ] **Glyph atlas** — text draws currently break geometry batches (z-order
flush per run); an atlas texture lets glyphs ride the vertex batches.
- [ ] **Retained layer textures** (later) — RenderNode-style caching of static
subtrees keyed by draw content.
## 3. Skia renderer
@@ -62,12 +69,18 @@ RSS; expect a plateau without GC sawtooth).
## 4. Correctness / parity gaps (future bug reports waiting)
Ordered by likelihood of a user hitting them:
Ordered by likelihood of a user hitting them. NOTE: this list predates several
fixes — verify a claim against the parity heatmaps before acting (gradients and
rotation both turned out already-working). In a parity `_diff.png`: a solid
bright block = real gap; ghosted text = font drift.
- [ ] **Gradient brushes on SDL** — `Brush.linearGradient/radialGradient`
render solid; the per-vertex sampler infrastructure is half-present.
- [ ] **Layer rotation** — rotationZ repositions but content doesn't rotate
(hit-testing too).
- [x] **Gradient brushes on SDL** — samplers were already implemented; the real
bug was UNDER-SAMPLING: fills sampled only corners, so radial/sweep on a
rect/round-rect rendered FLAT (a square's corners are equidistant from centre
→ identical t). Gradient fills now grid-mesh the interior (emitRectMesh);
radial/sweep match upstream. (Solid fills unchanged — one quad.)
- [x] **Layer rotation** — verified WORKING (demo GraphicsLayer "Rotation
(live)" + "Combined" rotate correctly); stale claim removed.
- [ ] **Real `saveLayer` alpha on SDL** — overlapping content composites at
paint level; needs an offscreen (the clip-target pool can serve it).
- [ ] **`clipPath` generic shapes** — bbox fallback clips square.
@@ -78,8 +91,9 @@ Ordered by likelihood of a user hitting them:
JVM from the same commonMain composables, pixel-diffs, ranks by %differ,
emits per-screen diff heatmaps + side-by-side compares (pct in filename).
build/parity/ (gitignored). See scripts/parity/README.md.
- [ ] Promote the press/hover automation rig (window-handle-relative input +
screenshot) into `scripts/`.
- [x] Promote the press/hover automation rig into `scripts/probe/`:
window-CLIENT-relative input (click/hover/hold, fractional coords, process-
addressed) + PrintWindow capture (works occluded). See scripts/probe/README.md.
## 6. Long-term
@@ -32,6 +32,7 @@ import com.compose.sdl.window.createPopupHostState
import kotlinx.coroutines.*
import kotlinx.coroutines.test.resetMain
import kotlinx.coroutines.test.setMain
import kotlinx.cinterop.toKString
import kotlinx.cinterop.reinterpret
import sdl3.SDL_Delay
import sdl3.SDL_GetPerformanceCounter
@@ -150,9 +151,8 @@ fun nativeComposeApp(content: @Composable ApplicationScope.() -> Unit) {
// Main loop
var vGcLastTicks = SDL_GetTicks()
var vRenderedSinceGc = false
val vProfiler = if (platform.posix.getenv("CDN_PROFILE") != null) FrameProfiler() else null
while (!runtime.exitRequested) {
vProfiler?.mark()
FrameProfiler.mark()
Snapshot.sendApplyNotifications()
// ============
@@ -180,7 +180,7 @@ fun nativeComposeApp(content: @Composable ApplicationScope.() -> Unit) {
}
mainDispatcher.drainPending()
vProfiler?.phase(0)
FrameProfiler.phase("events")
// ============
// App composition pump — Window()s may appear / disappear here.
@@ -200,7 +200,7 @@ fun nativeComposeApp(content: @Composable ApplicationScope.() -> Unit) {
// Exit when the last window is gone (after at least one existed).
if (runtime.hadWindow && runtime.windows.isEmpty()) runtime.exitRequested = true
if (runtime.exitRequested) break
vProfiler?.phase(1)
FrameProfiler.phase("app")
// ============
// Per-window pump + render.
@@ -214,16 +214,16 @@ fun nativeComposeApp(content: @Composable ApplicationScope.() -> Unit) {
Snapshot.sendApplyNotifications()
vW.frameClock.sendFrame()
yield()
vProfiler?.phase(2)
FrameProfiler.phase("pump")
if (vW.shouldRender()) {
vW.renderFrame()
vAnyRendered = true
if (!vW.backend.vsyncEnabled) vAllVsync = false
}
vProfiler?.phase(3)
FrameProfiler.phase("render")
}
runtime.reapDestroyed()
vProfiler?.frameDone(vAnyRendered)
FrameProfiler.frameDone(vAnyRendered)
// ============
// Drain deferred native-resource disposals on the MAIN thread —
@@ -280,41 +280,63 @@ fun nativeComposeApp(content: @Composable ApplicationScope.() -> Unit) {
SDL_Quit()
}
/* CDN_PROFILE=1 — per-phase main-loop timings, printed every ~2s of rendered
frames. Phases: events (poll+dispatch), app (app-composition pump), pump
(per-window clocks/dispatch), render (layout+draw+present). Measure first,
optimize second — see ROADMAP.md. */
/* CDN_PROFILE=1 — per-phase timings, printed every ~2s of rendered frames.
A named-phase SINGLETON so both the main loop (events / app / pump / render)
AND renderFrame's sub-steps (render.layout / render.draw / render.present)
report into one line. `mark()` resets the stopwatch; `phase(name)` charges
the elapsed since the last mark/phase to that name. Measure first, optimize
second — see ROADMAP.md. */
@OptIn(kotlinx.cinterop.ExperimentalForeignApi::class)
internal class FrameProfiler {
internal object FrameProfiler {
// null until first checked; then true/false for the run's lifetime.
private var fEnabled: Boolean? = null
// Output file — resolved once from CDN_PROFILE. Writing to a file (not
// stdout) lets GUI-subsystem apps (the demo links --subsystem,windows, so
// it has no console) be profiled too. CDN_PROFILE=1 → "cdn_profile.log" in
// the cwd; CDN_PROFILE=<path> → that path.
private var fPath: String = "cdn_profile.log"
val enabled: Boolean
get() = fEnabled ?: run {
val vEnv = platform.posix.getenv("CDN_PROFILE")?.toKString()
if (vEnv != null && vEnv != "1" && vEnv.isNotEmpty()) fPath = vEnv
(vEnv != null).also { fEnabled = it }
}
private val fFreq = SDL_GetPerformanceFrequency().toDouble()
private val fSum = DoubleArray(4)
private val fMax = DoubleArray(4)
// Insertion-ordered so the printed line follows the call order.
private val fSum = LinkedHashMap<String, Double>()
private val fMax = LinkedHashMap<String, Double>()
private var fFrames = 0
private var fLastPrintMs = SDL_GetTicks()
private var fMark = 0uL
fun mark() { fMark = SDL_GetPerformanceCounter() }
fun mark() { if (enabled) fMark = SDL_GetPerformanceCounter() }
fun phase(inIndex: Int) {
fun phase(inName: String) {
if (!enabled) return
val vNow = SDL_GetPerformanceCounter()
val vMs = (vNow - fMark).toDouble() * 1000.0 / fFreq
fSum[inIndex] += vMs
if (vMs > fMax[inIndex]) fMax[inIndex] = vMs
fSum[inName] = (fSum[inName] ?: 0.0) + vMs
if (vMs > (fMax[inName] ?: 0.0)) fMax[inName] = vMs
fMark = vNow
}
fun frameDone(inRendered: Boolean) {
if (!enabled) return
if (inRendered) fFrames++
val vNowMs = SDL_GetTicks()
if (vNowMs - fLastPrintMs >= 2000u && fFrames > 0) {
val vNames = listOf("events", "app", "pump", "render")
val vParts = vNames.mapIndexed { vI, vName ->
val vAvg = fSum[vI] / fFrames
"$vName=${(vAvg * 100).toInt() / 100.0}/${(fMax[vI] * 100).toInt() / 100.0}ms"
val vParts = fSum.keys.map { vName ->
val vAvg = (fSum[vName] ?: 0.0) / fFrames
"$vName=${(vAvg * 100).toInt() / 100.0}/${((fMax[vName] ?: 0.0) * 100).toInt() / 100.0}ms"
}
val vTotal = fSum.sum() / fFrames
println("[profile] frames=$fFrames avg/max " + vParts.joinToString(" ") + " total=${(vTotal * 100).toInt() / 100.0}ms")
for (vI in fSum.indices) { fSum[vI] = 0.0; fMax[vI] = 0.0 }
val vLine = "[profile] frames=$fFrames avg/max " + vParts.joinToString(" ") + "\n"
val vFile = platform.posix.fopen(fPath, "a")
if (vFile != null) {
platform.posix.fputs(vLine, vFile)
platform.posix.fclose(vFile)
}
fSum.clear(); fMax.clear()
fFrames = 0
fLastPrintMs = vNowMs
}
@@ -734,6 +756,7 @@ internal class WindowInstance(
vRender.ensureSize(backend.pixelWidth, backend.pixelHeight)
host.setConstraints(backend.pixelWidth, backend.pixelHeight)
host.measureAndLayout()
FrameProfiler.phase(" layout")
// Hover refresh after layout (upstream skiko: SyntheticEventSender).
if (hasMousePos) {
@@ -746,7 +769,9 @@ internal class WindowInstance(
// Probe consumers end the app when their scenario completes.
facade.close()
}
FrameProfiler.phase(" draw")
vRender.endFrame()
FrameProfiler.phase(" present")
frameIndex++
// FPS — refreshed ~once a second, per window.
@@ -243,7 +243,10 @@ internal class Sdl3DrawScope(
val vB = vT + size.height
when (style) {
Fill -> {
emitQuad(vL, vT, vR, vT, vR, vB, vL, vB, vSampler)
// Gradient fills need interior samples (see emitRectMesh); solid
// fills are a single flat quad.
if (brush is SolidColor) emitQuad(vL, vT, vR, vT, vR, vB, vL, vB, vSampler)
else emitRectMesh(vL, vT, vR, vB, vSampler)
// Axis-aligned rects are pixel-crisp and need no AA. When the affine
// rotates/shears them, the edges become diagonal — feather each edge
// outward (normals in local space; the matrix orients them on screen).
@@ -524,20 +527,30 @@ internal class Sdl3DrawScope(
val vY = fOriginY + topLeft.y
val vW = size.width
val vH = size.height
// Gradient fills sample per-vertex, so the interior must be meshed
// (corner-only sampling renders radial/sweep flat — see emitRectMesh);
// solid fills stay single quads.
val vGrad = brush !is SolidColor
if (vR <= 0f) {
// Trivial: just two triangles.
emitQuad(vX, vY, vX + vW, vY, vX + vW, vY + vH, vX, vY + vH, vSampler)
if (vGrad) emitRectMesh(vX, vY, vX + vW, vY + vH, vSampler)
else emitQuad(vX, vY, vX + vW, vY, vX + vW, vY + vH, vX, vY + vH, vSampler)
return
}
// Body in 3 strips: middle (full width × inner height), top edge,
// bottom edge — plus the 4 corner arcs.
if (style == Fill) {
if (vGrad) {
emitRectMesh(vX, vY + vR, vX + vW, vY + vH - vR, vSampler) // middle
emitRectMesh(vX + vR, vY, vX + vW - vR, vY + vR, vSampler) // top edge
emitRectMesh(vX + vR, vY + vH - vR, vX + vW - vR, vY + vH, vSampler) // bottom edge
} else {
// Middle strip
emitQuad(vX, vY + vR, vX + vW, vY + vR, vX + vW, vY + vH - vR, vX, vY + vH - vR, vSampler)
// Top edge (between left+right corners)
emitQuad(vX + vR, vY, vX + vW - vR, vY, vX + vW - vR, vY + vR, vX + vR, vY + vR, vSampler)
// Bottom edge
emitQuad(vX + vR, vY + vH - vR, vX + vW - vR, vY + vH - vR, vX + vW - vR, vY + vH, vX + vR, vY + vH, vSampler)
}
// 4 corner fills — segment count adapts to the corner radius.
val vSeg = arcSegments(90f, vR)
emitFilledArc(vX + vR, vY + vR, vR, vR, 180f, 90f, false, vSeg, vSampler)
@@ -854,6 +867,34 @@ internal class Sdl3DrawScope(
emitTri(ax, ay, cx, cy, dx, dy, inSampler)
}
// Axis-aligned rect subdivided into a grid, so a per-vertex sampler is
// evaluated across the INTERIOR, not just the 4 corners. Corner-only
// sampling renders radial/sweep gradients flat (a square's corners are all
// equidistant from its centre → identical t → solid fill) and clips
// non-uniform multi-stop linear ramps. SDL interpolates linearly within
// each cell, so ~16 cells/axis tracks the gradient curve closely. Solid
// fills never come here (they use emitQuad — one quad).
private fun emitRectMesh(
inL: Float, inT: Float, inR: Float, inB: Float, inSampler: Sampler,
) {
// ~one cell per 8 device px per axis (1..24), so thin edge strips stay
// cheap while large fills get enough interior samples for the gradient.
val vCols = (((inR - inL) / 8f).toInt()).coerceIn(1, 24)
val vRows = (((inB - inT) / 8f).toInt()).coerceIn(1, 24)
val vW = (inR - inL) / vCols
val vH = (inB - inT) / vRows
for (vRow in 0 until vRows) {
val vY0 = inT + vRow * vH
val vY1 = if (vRow == vRows - 1) inB else vY0 + vH
for (vCol in 0 until vCols) {
val vX0 = inL + vCol * vW
val vX1 = if (vCol == vCols - 1) inR else vX0 + vW
emitTri(vX0, vY0, vX1, vY0, vX1, vY1, inSampler)
emitTri(vX0, vY0, vX1, vY1, vX0, vY1, inSampler)
}
}
}
private fun emitTri(
ax: Float, ay: Float, bx: Float, by: Float, cx: Float, cy: Float,
inSampler: Sampler,
+35
View File
@@ -0,0 +1,35 @@
# Probe driver
Launches a native compose-desktop-native app window, sends **window-relative**
synthetic input, and captures the client area — for reproducing visual bugs
deterministically (this is the packaged form of the rigs that caught the
square-on-click and TLS-chain regressions).
Why window-relative + PrintWindow: input coordinates are FRACTIONS of the
window client rect and addressed by process name, so a probe doesn't depend on
where the window landed or on it being foreground; capture uses `PrintWindow`,
which grabs the window even when occluded or unfocused (a plain screen grab
would catch whatever's on top).
```bash
# screenshot a demo screen
python scripts/probe/probe.py demo --screen=Images --shot images.png
# press-and-hold a point (fraction of the window) and capture DURING the press
python scripts/probe/probe.py demo --screen=Buttons --hold 0.1,0.32 --shot press.png
# hover then capture
python scripts/probe/probe.py demo --screen=Shapes --hover 0.3,0.45 --shot hov.png
# an app built elsewhere (e.g. the bridge example)
python scripts/probe/probe.py shared --exe /path/to/shared.exe --click 0.5,0.5 --shot x.png
```
- `proc` = the window's process name (`demo`, `apidemo`, `shared`).
- Actions run in order; `--hold` captures mid-press (async), `--click` /
`--hover` complete first. `--shot` captures after `--settle` seconds.
- Windows-only (uses Win32 input + PrintWindow). Pillow only needed if you
post-process the PNG.
Pairs with `scripts/parity/` (whole-screen diffing); this is for targeted
interaction repro.
+28
View File
@@ -0,0 +1,28 @@
param([string]$proc, [string]$out)
# Capture a window's CLIENT area by process name, via PrintWindow — works even
# when the window is occluded or not focused (unlike a screen grab). Used by the
# probe driver to screenshot native app windows deterministically.
Add-Type -AssemblyName System.Drawing
Add-Type @"
using System;
using System.Runtime.InteropServices;
public class Cap {
[DllImport("user32.dll")] public static extern bool GetClientRect(IntPtr h, out RECT r);
[DllImport("user32.dll")] public static extern bool PrintWindow(IntPtr h, IntPtr dc, uint flags);
public struct RECT { public int L, T, R, B; }
}
"@
$h = (Get-Process $proc -ErrorAction SilentlyContinue | Where-Object { $_.MainWindowHandle -ne 0 } | Select-Object -First 1).MainWindowHandle
if (-not $h -or $h -eq [IntPtr]::Zero) { Write-Output "NO WINDOW"; exit 1 }
$r = New-Object Cap+RECT
[Cap]::GetClientRect($h, [ref]$r) | Out-Null
$w = $r.R - $r.L; $ht = $r.B - $r.T
if ($w -le 0 -or $ht -le 0) { Write-Output "ZERO SIZE"; exit 1 }
$bmp = New-Object System.Drawing.Bitmap $w, $ht
$g = [System.Drawing.Graphics]::FromImage($bmp)
$hdc = $g.GetHdc()
# flags=3 : PW_CLIENTONLY | PW_RENDERFULLCONTENT (captures accelerated content)
[Cap]::PrintWindow($h, $hdc, 3) | Out-Null
$g.ReleaseHdc($hdc)
$bmp.Save($out, [System.Drawing.Imaging.ImageFormat]::Png)
Write-Output "captured ${w}x${ht}"
+42
View File
@@ -0,0 +1,42 @@
param(
[string]$proc, # process name of the target window
[string]$action, # click | hold | hover | move
[double]$fx, # fractional X within the window client area (0..1)
[double]$fy, # fractional Y
[int]$holdMs = 80 # press duration for 'hold'
)
# Window-CLIENT-relative synthetic input against a native app window, addressed
# by process name — so probes don't depend on where the window landed or on it
# being the foreground window (it's raised to foreground first). Coordinates are
# FRACTIONS of the client rect, so they're resolution/DPI independent.
Add-Type @"
using System;
using System.Runtime.InteropServices;
public class In {
[DllImport("user32.dll")] public static extern bool GetClientRect(IntPtr h, out RECT r);
[DllImport("user32.dll")] public static extern bool ClientToScreen(IntPtr h, ref POINT p);
[DllImport("user32.dll")] public static extern bool SetForegroundWindow(IntPtr h);
[DllImport("user32.dll")] public static extern bool SetCursorPos(int x, int y);
[DllImport("user32.dll")] public static extern void mouse_event(uint f, uint x, uint y, uint d, UIntPtr e);
public struct RECT { public int L, T, R, B; }
public struct POINT { public int X, Y; }
public const uint DOWN = 2, UP = 4;
}
"@
$h = (Get-Process $proc -ErrorAction SilentlyContinue | Where-Object { $_.MainWindowHandle -ne 0 } | Select-Object -First 1).MainWindowHandle
if (-not $h -or $h -eq [IntPtr]::Zero) { Write-Output "NO WINDOW"; exit 1 }
$r = New-Object In+RECT
[In]::GetClientRect($h, [ref]$r) | Out-Null
$p = New-Object In+POINT
$p.X = [int](($r.R - $r.L) * $fx); $p.Y = [int](($r.B - $r.T) * $fy)
[In]::ClientToScreen($h, [ref]$p) | Out-Null
[In]::SetForegroundWindow($h) | Out-Null
Start-Sleep -Milliseconds 120
[In]::SetCursorPos($p.X, $p.Y) | Out-Null
switch ($action) {
"move" { }
"hover" { Start-Sleep -Milliseconds 250 }
"click" { Start-Sleep -Milliseconds 120; [In]::mouse_event([In]::DOWN,0,0,0,[UIntPtr]::Zero); Start-Sleep -Milliseconds 60; [In]::mouse_event([In]::UP,0,0,0,[UIntPtr]::Zero) }
"hold" { Start-Sleep -Milliseconds 120; [In]::mouse_event([In]::DOWN,0,0,0,[UIntPtr]::Zero); Start-Sleep -Milliseconds $holdMs; [In]::mouse_event([In]::UP,0,0,0,[UIntPtr]::Zero) }
}
Write-Output "$action at $($p.X),$($p.Y)"
+101
View File
@@ -0,0 +1,101 @@
#!/usr/bin/env python3
"""
Drive a native compose-desktop-native app window for manual/automated visual
checks: launch it, send window-relative input (click / hover / hold / move),
and capture the client area via PrintWindow (works even when occluded).
Built from the ad-hoc rigs used to reproduce the square-on-click and TLS-chain
bugs — window-CLIENT-relative and process-addressed, so it doesn't care where
the window lands or whether it's focused.
Examples:
# boot the demo's Images screen, screenshot after 4s
python scripts/probe/probe.py demo --screen=Images --shot=images.png
# boot bubble-wrap (example repo), press-and-hold a bubble mid-sheet, capture
python scripts/probe/probe.py shared --exe <path-to>/shared.exe \\
--hold 0.55,0.55 --shot press.png
# hover then capture
python scripts/probe/probe.py demo --screen=Shapes --hover 0.3,0.4 --shot hov.png
`proc` is the window's process name (demo / apidemo / shared). Native apps
built here use the mingwX64 debug exe by default; override with --exe. Actions
run in order; --shot captures after them. Windows-only. Needs Pillow only if
you pass --crop.
"""
import argparse, subprocess, sys, time, os, signal
from pathlib import Path
REPO = Path(__file__).resolve().parents[2]
HERE = Path(__file__).resolve().parent
DEFAULT_EXE = {
"demo": REPO / "demo/build/bin/mingwX64/debugExecutable/demo.exe",
"apidemo": REPO / "apidemo/build/bin/mingwX64/debugExecutable/apidemo.exe",
}
def ps(script: str, *args: str):
return subprocess.run(
["powershell", "-ExecutionPolicy", "Bypass", "-File", str(HERE / script), *args],
capture_output=True, text=True,
)
def frac(s: str):
x, y = s.split(",")
return float(x), float(y)
def main():
ap = argparse.ArgumentParser()
ap.add_argument("proc", help="window process name (demo / apidemo / shared)")
ap.add_argument("--exe", help="path to the executable (default: mingwX64 debug for demo/apidemo)")
ap.add_argument("--screen", help="pass --screen=<Name> to the app (demo)")
ap.add_argument("--args", default="", help="extra args passed verbatim to the exe")
ap.add_argument("--boot", type=float, default=4.0, help="seconds to wait after launch")
ap.add_argument("--click", help="fractional X,Y to click (e.g. 0.5,0.5)")
ap.add_argument("--hold", help="fractional X,Y to press-and-hold")
ap.add_argument("--hover", help="fractional X,Y to hover")
ap.add_argument("--hold-ms", type=int, default=900, help="hold duration")
ap.add_argument("--settle", type=float, default=0.3, help="seconds between action and capture")
ap.add_argument("--shot", help="capture the window to this PNG after the actions")
args = ap.parse_args()
exe = Path(args.exe) if args.exe else DEFAULT_EXE.get(args.proc)
if not exe or not exe.exists():
print(f"exe not found: {exe} (build it, or pass --exe)", file=sys.stderr)
return 1
cmd = [str(exe)]
if args.screen:
cmd.append(f"--screen={args.screen}")
if args.args:
cmd += args.args.split()
proc = subprocess.Popen(cmd)
try:
time.sleep(args.boot)
if args.hover:
fx, fy = frac(args.hover)
print(ps("_input.ps1", "-proc", args.proc, "-action", "hover", "-fx", str(fx), "-fy", str(fy)).stdout.strip())
if args.click:
fx, fy = frac(args.click)
print(ps("_input.ps1", "-proc", args.proc, "-action", "click", "-fx", str(fx), "-fy", str(fy)).stdout.strip())
if args.hold:
fx, fy = frac(args.hold)
# hold runs async so we can capture DURING the press
p = subprocess.Popen(["powershell", "-ExecutionPolicy", "Bypass", "-File", str(HERE / "_input.ps1"),
"-proc", args.proc, "-action", "hold", "-fx", str(fx), "-fy", str(fy), "-holdMs", str(args.hold_ms)])
time.sleep(args.settle)
if args.shot:
out = str(Path(args.shot).resolve())
r = ps("_capture.ps1", "-proc", args.proc, "-out", out)
print(f"{r.stdout.strip()} -> {out}")
finally:
subprocess.run(["taskkill", "/F", "/IM", exe.name], capture_output=True)
return 0
if __name__ == "__main__":
sys.exit(main())