# Scanner performance: database loading, re-scans, TLSH in SQL, Aho-Corasick

> Database load time dropped from 60–77 s to ~11 s, a re-scan of Numbers.app (33819 files) from 131.7 s to 24.7 s, and moving TLSH candidate selection into SQL gave identical results (718/718) ~1.6× faster; `ahocorasick_rs` built lazily beats loading a pickled `pyahocorasick` automaton, and threaded sharding gives no speedup because of the GIL.

Area: Security (macOS malware scanning). Page: https://benchmarks.site/scanner-performance/. Methodology and caveats: https://benchmarks.site/methodology/#malware

These are manual measurements taken on the development Mac in November 2026. There are no raw result
files for them; the numbers below are the recorded results.

## Ours (own engine)

| Measurement | Date | Before | After | Result |
|---|---|---|---|---|
| Database load time | 2026-11-09 | 60–77 s | ~11 s | about 5.5–7× faster |
| Scan memory: re-scan of Numbers.app (33819 files) | 2026-11-11 | 131.7 s | 24.7 s | about 5.3× faster |
| TLSH candidate restriction in SQL (`tlsh_candidates_within`) | 2026-11-09 | — | — | 718/718 identical results, ~1.6× faster |

- **Database load time:** time to load the signature and hash databases at engine start.
- **Scan memory:** the scanner remembers files it has already scanned, so a second scan of the same
  application bundle (Numbers.app, 33819 files) only has to handle what changed.
- **TLSH in SQL:** the set of TLSH fuzzy-hash candidates within a distance limit is restricted already in
  the SQL query (`tlsh_candidates_within`); all 718 compared results were identical to the previous method.

## External libraries

| Library / approach | Date | Measurement | Result |
|---|---|---|---|
| `pyahocorasick` (save/load of a prebuilt automaton, pickle) | 2026-11-09 | load time | 76 s |
| `ahocorasick_rs` (build the automaton) | 2026-11-09 | build time | ~25 s |
| Building the automaton in threads (sharding) | 2026-11-09 | build time | no speedup (GIL) |

- Loading a pickled `pyahocorasick` automaton (76 s) was slower than building one from scratch with
  `ahocorasick_rs` (~25 s), so `ahocorasick_rs` stays, and the automaton is built lazily (only when it is
  first needed).
- Splitting the pattern set into shards and building them in parallel threads gave no speedup, because
  of Python's global interpreter lock (GIL).
