146 lines
4.3 KiB
Markdown
146 lines
4.3 KiB
Markdown
|
## Request throttling
|
|||
|
|
|
|||
|
|
Debounce and cancel high-frequency server calls
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Summary
|
|||
|
|
|
|||
|
|
Some user interactions trigger bursts of server requests that can overlap and return out of order. We’ll debounce frequent triggers and cancel in-flight requests (or ignore stale results) for file search and LSP refresh.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Goals
|
|||
|
|
|
|||
|
|
- Reduce redundant calls from file search and LSP refresh
|
|||
|
|
- Prevent stale responses from overwriting newer UI state
|
|||
|
|
- Preserve responsive typing and scrolling during high activity
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Non-goals
|
|||
|
|
|
|||
|
|
- Changing server-side behavior or adding new endpoints
|
|||
|
|
- Implementing global request queues for all SDK calls
|
|||
|
|
- Persisting search results across reloads
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Current state
|
|||
|
|
|
|||
|
|
- File search calls `sdk.client.find.files` via `files.searchFilesAndDirectories`.
|
|||
|
|
- LSP refresh is triggered frequently (exact call sites vary, but the refresh behavior is high-frequency).
|
|||
|
|
- Large UI modules involved include `packages/app/src/pages/layout.tsx` and `packages/app/src/components/prompt-input.tsx`.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Proposed approach
|
|||
|
|
|
|||
|
|
- Add a small request coordinator utility:
|
|||
|
|
- debounced triggering (leading/trailing configurable)
|
|||
|
|
- cancellation via `AbortController` when supported
|
|||
|
|
- stale-result protection via monotonic request ids when abort is not supported
|
|||
|
|
- Integrate coordinator into:
|
|||
|
|
- `files.searchFilesAndDirectories` (wrap `sdk.client.find.files`)
|
|||
|
|
- LSP refresh call path (wrap refresh invocation and ensure only latest applies)
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Phased implementation steps
|
|||
|
|
|
|||
|
|
1. Add a debounced + cancellable helper
|
|||
|
|
|
|||
|
|
- Create `packages/app/src/utils/requests.ts` with:
|
|||
|
|
- `createDebouncedAsync(fn, delayMs)`
|
|||
|
|
- `createLatestOnlyAsync(fn)` that drops stale responses
|
|||
|
|
- Prefer explicit, readable primitives over a single complex abstraction.
|
|||
|
|
|
|||
|
|
Sketch:
|
|||
|
|
|
|||
|
|
```ts
|
|||
|
|
function createLatestOnlyAsync<TArgs extends unknown[], TResult>(
|
|||
|
|
fn: (args: { input: TArgs; signal?: AbortSignal }) => Promise<TResult>,
|
|||
|
|
) {
|
|||
|
|
let id = 0
|
|||
|
|
let controller: AbortController | undefined
|
|||
|
|
|
|||
|
|
return async (...input: TArgs) => {
|
|||
|
|
id += 1
|
|||
|
|
const current = id
|
|||
|
|
controller?.abort()
|
|||
|
|
controller = new AbortController()
|
|||
|
|
|
|||
|
|
const result = await fn({ input, signal: controller.signal })
|
|||
|
|
if (current !== id) return
|
|||
|
|
return result
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
2. Apply to file search
|
|||
|
|
|
|||
|
|
- Update `files.searchFilesAndDirectories` to:
|
|||
|
|
- debounce input changes (e.g. 150–300 ms)
|
|||
|
|
- abort prior request when a new query begins
|
|||
|
|
- ignore results if they are stale
|
|||
|
|
- Ensure “empty query” is handled locally without calling the server.
|
|||
|
|
|
|||
|
|
3. Apply to LSP refresh
|
|||
|
|
|
|||
|
|
- Identify the refresh trigger points used during typing and file switching.
|
|||
|
|
- Add:
|
|||
|
|
- debounce for rapid triggers (e.g. 250–500 ms)
|
|||
|
|
- cancellation for in-flight refresh if supported
|
|||
|
|
- last-write-wins behavior for applying diagnostics/results
|
|||
|
|
|
|||
|
|
4. Add feature flags and metrics
|
|||
|
|
|
|||
|
|
- Add flags:
|
|||
|
|
- `requests.debounce.fileSearch`
|
|||
|
|
- `requests.latestOnly.lspRefresh`
|
|||
|
|
- Add simple dev-only counters for “requests started / aborted / applied”.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Data migration / backward compatibility
|
|||
|
|
|
|||
|
|
- No persisted data changes.
|
|||
|
|
- Behavior is compatible as long as UI state updates only when the “latest” request resolves.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Risk + mitigations
|
|||
|
|
|
|||
|
|
- Risk: aggressive debounce makes UI feel laggy.
|
|||
|
|
- Mitigation: keep delays small and tune separately for search vs refresh.
|
|||
|
|
- Risk: aborting requests may surface as errors in logs.
|
|||
|
|
- Mitigation: treat `AbortError` as expected and do not log it as a failure.
|
|||
|
|
- Risk: SDK method may not accept `AbortSignal`.
|
|||
|
|
- Mitigation: use request-id stale protection even without true cancellation.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Validation plan
|
|||
|
|
|
|||
|
|
- Manual scenarios:
|
|||
|
|
- type quickly in file search and confirm requests collapse and results stay correct
|
|||
|
|
- trigger LSP refresh repeatedly and confirm diagnostics do not flicker backward
|
|||
|
|
- Add a small unit test for latest-only behavior (stale results are ignored).
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Rollout plan
|
|||
|
|
|
|||
|
|
- Ship helpers behind flags default off.
|
|||
|
|
- Enable file search debounce first (high impact, easy to validate).
|
|||
|
|
- Enable LSP latest-only next, then add cancellation if SDK supports signals.
|
|||
|
|
- Keep a quick rollback by disabling the flags.
|
|||
|
|
|
|||
|
|
---
|
|||
|
|
|
|||
|
|
### Open questions
|
|||
|
|
|
|||
|
|
- Does `sdk.client.find.files` accept an abort signal today, or do we need stale-result protection only?
|
|||
|
|
- Where is LSP refresh initiated, and does it have a single chokepoint we can wrap?
|
|||
|
|
- What debounce values feel best for common repos and slower machines?
|