Obscura Trace’s requirements call for handling one million files and ten terabytes on an ordinary Mac. Before publishing 0.2.0, we had the app analyze one million real photographs. Here is what holds, and what does not hold yet.
All the measurements below were taken on October 9, 2026 on an Intel Core i9-9880H MacBook Pro, 64 GiB of memory, internal SSD, release build. Apple Silicon has not been measured yet for the real inventory.
The collection: one million real JPEGs
These are not empty files. The collection is made of 10⁶ real JPEGs written by ImageIO, 2.65 GB in total, with the makeup of a real photo library:
- 80% originals;
- 10% exact copies;
- 10% visual duplicates.
Each measurement runs in a fresh process. The memory peak is read from the kernel, not sampled.
What holds
The full analysis runs to the end. The five phases follow each other: file walk, metadata, hashing, image analysis, similarity index. The million files are inventoried, with a peak of 659 MiB, under the 2 GiB ceiling we set ourselves. It took 52 minutes, on a machine shared with other workloads.
Moves are found exactly. On a volume without stable file identifiers, like an exFAT drive or an SMB share, every file was moved to another folder.
- Without full checksums: 990,054 match proposals, exactly the expected number.
- With full checksums: 800,000 files matched by their content. The 100,000 ambiguous cases, two identical copies for a single missing file, were never matched at random.
The manifest is faithful. The manifest of a one-million-file transfer is written with flat memory: 29.6 MiB, versus 28.5 MiB for one hundred thousand. Read back, it contains no SHA-256 that differs from the journal.
A quadratic defect was found and fixed. Ten thousand all-black images all share the same visual fingerprint. Comparing them pairwise cost 1.5 × 10⁸ comparisons, versus a single one now. On the million files, the last phase of the analysis went from over an hour (stopped) to 42 seconds.
The release suite: zero silent loss
Every release goes through a suite of ten scenarios before publication. For 0.2.0, all ten pass:
- inventory of 10⁶ entries;
- 10 TB archive;
- crash and resume;
- unplugged drive;
- duplicate detection;
- copy integrity;
- name collisions;
- catalog migration;
- architecture boundaries;
- decision audit.
Silent losses: 0.
Two honest caveats. The suite’s inventory is synthetic: the analysis above complements it with real files. The 10 TB are declared to the catalog and the plan, not copied byte by byte.
What does not hold yet
Our internal targets require memory not to grow with the size of the collection: a peak at 10⁶ at most three times the one at 10⁵. Several measurements exceed that ratio.
| Measurement | 10⁵ | 10⁶ | Ratio |
|---|---|---|---|
| Full analysis | 142 MiB | 659 MiB | 4.6 |
| Reading back a manifest | 327 MiB | 2,871 MiB | 8.8 |
| Batch-rename preview | 170 MiB | 1,181 MiB | 7.0 |
| Move matching | 215 MiB | 1,376 MiB | 6.4 |
Results remain correct. But a very large collection needs more memory than it should. Each of these gaps has its ticket and will be fixed before version 1.0.
Another measurement revealed a defect. An analysis killed halfway and then resumed gives the same groups, the same members, and the same masters, with no contradictory checksum. Yet it re-read in full far more files than needed: 899,841 full SHA-256 checksums instead of 200,000. The cause is not the interruption: any second analysis re-reads unchanged files that share their size with another. The guide now says so, and the fix is in progress.
Why publish these numbers
Software that touches irreplaceable archives should not say “fast” or “reliable” without showing the measurement. These are the numbers of the release you download, limits included. They are summarized in the FAQ, current limitations, and glossary chapter of the guide and in the changelog.