Transferring and consolidating: the Centre de transfert

The Centre de transfert (Transfer Center) copies or moves the contents of a source to a destination. It is the only screen in the app that writes to your disks, and it does so only after a plan you have read and a confirmation you have approved.

The path is always the same:

Options ──► Plan (dry run or real) ──► Confirmation ──► Execution ──► Report
                                                           │
                                                           └──► Interruption ──► Resume

Execution can only be reached from the confirmation of a real plan. A dry run is never executed.

Opening the Centre de transfert on a source

A transfer starts from a source that has already been analyzed. You open it from the end of an analysis, with the “Transférer « nom de la source »…” (Transfer “source name”…) button of the Assistant de scan (Scan Assistant) (see chapter 3), or directly from the sidebar.

Opened from the sidebar with no transfer being prepared, the Centre de transfert shows “Aucun transfert en préparation” (No transfer being prepared), then the list of sources already analyzed — their name, their number of files and the path where they are mounted (or “Chemin inconnu : le volume de la source n’est pas joignable” (Unknown path: the source’s volume is unreachable)) — each with a “Préparer un transfert” (Prepare a transfer) button. This button opens the options for that source without running an analysis again: the plan starts from the inventory of the last analysis. If a source has changed since, run its analysis again first (only new or modified files are reread): the Assistant reuses the types and exclusions of its last analysis; an XMP inventoried the first time still is; if it differs, the summary flags it.

At the top of the screen: “Source : nom” (Source: name) and the path where the source is currently mounted.

The Centre de transfert open on a source: no destination yet; “Choisir…” (Choose…) opens the folder picker.

1. Options

The “Options du transfert” (Transfer options) card: chosen destination, mode, collision strategy, dry run or real plan.

The “Options du transfert” (Transfer options) card contains:

• “Destination” (Destination) and the “Choisir…” (Choose…) button: choose an existing folder (“Dossier de destination du transfert” (Transfer destination folder)). The panel does not create folders; create it beforehand in the Finder if needed.
• “Mode” (Mode): “Copier” (Copy) (the default; sources stay in place) or “Déplacer” (Move) (verified copy, then removal of the sources).
• “Collision” (Collision): what happens if a file with the same name already exists at the destination:

Changing strategy withdraws the consent to replacement: it never carries over to another strategy. With “Remplacer (REPLACE)” (Replace), the computation buttons stay grayed out until the consent box is checked (“Le plan se calcule une fois le consentement donné.” (The plan is computed once consent is given.)).

The first suffix of “Renommer” (Rename) is _2: the copy is the second instance of the name, as in the Finder (IMG_0002.jpg → IMG_0002_2.jpg).

The destination must exist at the time of the plan, and can be neither inside the analyzed source, nor contain it, nor be the source itself. Otherwise the plan is not created: “Destination introuvable” (Destination not found) (disk unplugged, folder to be created), “Destination et source se recouvrent” (Destination and source overlap) or “La destination est la source” (The destination is the source).

• In “Déplacer” (Move) mode, what happens to the sources:
  • if permanent deletion is disabled in the Réglages (Settings) (the default case), the screen states: “Les sources seront mises à la Corbeille après vérification de leur copie. La suppression définitive est désactivée dans les Réglages.” (The sources will be moved to the Trash after their copy is verified. Permanent deletion is disabled in Settings.);
  • if it is allowed, a “Sources” (Sources) choice offers “Mettre les sources à la Corbeille” (Move the sources to the Trash) or “Supprimer définitivement les sources” (Permanently delete the sources). This second choice requires checking “Je consens à la suppression définitive des sources de ce transfert, une fois leur copie vérifiée.” (I consent to the permanent deletion of this transfer’s sources, once their copy is verified.)

What is transferred. All inventoried files present in the source, keeping the source’s folder tree under the destination folder. A main file and its companions (a RAW and its XMP, for example) form a unit: they go together or not at all, whatever the strategy. Decisions from the Centre de revue (Review Center) filter the plan: a file marked “doublon” (duplicate) (when a present master exists) is neither copied nor moved, and stays in place in the source, even in Move mode. The plan announces it on the line “Exclus par la revue (restent dans la source)” (Excluded by review (stay in the source)), and the confirmation repeats it. A group you have not reviewed, or that you have ignored, is transferred in full; a file “à décider” (to decide) is transferred. An excluded RAW keeps its XMP with it, in the source.

What the plan does not cover, and says so. Below the plan’s counts, one warning line per case: folders that were unreadable at the source’s last analysis (“N dossier(s) non lisible(s) à la dernière analyse : chemin. Leur contenu n’est pas dans ce plan.” (N unreadable folder(s) at the last analysis: path. Their content is not in this plan.)), files outside the rules of the last analysis (“N fichier(s) hors des règles de la dernière analyse, non planifié(s) : ils restent dans la source.” (N file(s) outside the rules of the last analysis, not planned: they stay in the source.)), and files located in a Photos, Lightroom… library (“N fichier(s) dans une bibliothèque (Photos, Lightroom…), jamais transféré(s) : ils restent dans la source.” (N file(s) in a library (Photos, Lightroom…), never transferred: they stay in the source.)). Before reformatting a disk you moved, check that none of these lines appears.

Caution: a file that the analysis did not inventory — a type not checked in step 2 (XMP, videos, PDF…) or a folder excluded in step 3 — is not transferred. Before consolidating a source, check that the analysis covered every type you want to take along. The app does not offer path templates (for example, filing by year); the command line does (obscura plan --template, see chapter 8).

Two buttons:

• “Simuler (dry run)” (Simulate (dry run)): computes the plan without being able to execute anything.
• “Calculer le plan” (Compute plan): computes a real plan, executable after confirmation.

They stay grayed out until a destination is chosen (and, with REPLACE, until replacement is consented to). During computation: “Simulation du plan…” (Simulating plan…) or “Calcul du plan…” (Computing plan…). Computing a plan transfers nothing: the app reads the source and the destination, and writes only to the catalog.

A dry run: the plan is computed and displayed, but it cannot be executed.

2. The plan

The real plan: the counts per category, then the list of operations, source path → destination path.

The card is titled “Simulation — rien ne sera exécuté” (Dry run — nothing will be executed) or “Plan réel” (Real plan). It shows “Destination finale : …” (Final destination: …) and the counts:

Below, the list of operations: “source → destination”, their status (“À copier” (To copy), “À déplacer” (To move), “Ignoré” (Skipped), “En revue” (In review)), any conflict and its reason, for example “contenu identique déjà présent (SHA-256)” (identical content already present (SHA-256)) or “contenu différent déjà présent à destination : IMG_0042.CR2” (different content already present at destination: IMG_0042.CR2). For a large plan, only the first 200 operations are read and listed (“Premières opérations (200 sur N)” (First operations (200 of N))); the counts cover the whole plan. The plan, its display and its execution never keep all the operations in memory: a collection of one million files is planned and transferred with the memory of a collection of one hundred thousand.

Buttons:

• “Modifier les options” (Change options) returns to the options;
• after a dry run, “Créer le plan réel” (Create real plan): computes the real plan with the same options and compares it with the dry run:
  • “Identique à la simulation : le plan réel fera exactement ce que la simulation annonçait.” (Identical to the dry run: the real plan will do exactly what the dry run announced.)
  • “Différent de la simulation : la source ou la destination ont changé depuis. Relisez le plan avant de confirmer.” (Different from the dry run: the source or the destination has changed since. Reread the plan before confirming.)
• for a real plan, “Vérifier et confirmer…” (Review and confirm…) opens the confirmation.

Disk space. While computing the plan, the app checks that the destination has enough room: the amount to write plus a margin of 5% of that amount, at least 64 MiB (67.1 MB) and at most 1 GiB (1.07 GB). A small transfer to a nearly full card or USB stick is therefore possible as soon as the card offers the amount to write plus 67.1 MB. Otherwise: “Espace insuffisant à destination” (Insufficient space at destination), with the three numbers in the same unit as the Finder (“Le plan demande 17,1 Mo, plus une marge de 67,1 Mo…” (The plan requires 17.1 MB, plus a margin of 67.1 MB…)), no plan is kept and nothing is written to the destination.

3. Confirmation

The confirmation: what will be written, its reversibility, and the recoverable and freed space, before “Exécuter le plan” (Execute plan).

The “Confirmation” (Confirmation) card summarizes what is going to be done — “N fichiers · X Go seront copiés vers …” (N files · X GB will be copied to …) (or “seront déplacés” (will be moved); in the singular, “sera copié” (will be copied)) — then one line per kind of operation, sorted by reversibility:

Then come the space measures, depending on what happens to the sources:

• a copy removes nothing: no space line;
• a move to the Trash: “Espace récupérable” (Recoverable space) — “Sources d’un déplacement. Mises à la Corbeille, elles occupent encore le disque jusqu’à ce que vous la vidiez ; l’application ne la vide jamais.” (Sources of a move. Once in the Trash, they still take up disk space until you empty it; the app never empties it.) — and “Espace libéré” (Freed space), at zero: “Seules les suppressions définitives libèrent de l’espace à l’exécution.” (Only permanent deletions free space at execution.);
• a permanent deletion: “Espace libéré” (Freed space) only — “Sources supprimées définitivement après vérification de leur copie : l’espace est libéré à l’exécution.” (Sources permanently deleted after their copy is verified: the space is freed at execution.) Nothing there is called “récupérable” (recoverable): nothing goes to the Trash.

If the plan contains operations in review, the confirmation announces it: “1 opération en revue ne sera pas exécutée : photos/IMG_0002.jpg (contenu différent déjà présent à destination : IMG_0002.jpg). Elle reste en place ; replanifiez après décision.” (1 operation in review will not be executed: photos/IMG_0002.jpg (different content already present at destination: IMG_0002.jpg). It stays in place; plan again after deciding.) (in the plural: “k opérations en revue ne seront pas exécutées” (k operations in review will not be executed)) The first three are named.

If the plan contains a permanent deletion, a red warning is shown (“Irréversible : … Rien ne permettra de les retrouver.” (Irreversible: … Nothing will let you get them back.)) and you must check “Je reconnais que cette opération est irréversible.” (I acknowledge that this operation is irreversible.) The execute button then takes the form of the irreversible button, with the number of files and the amount of data shown in the button.

Buttons: “Revenir au plan” (Back to plan) (the acknowledgment is forgotten) and “Exécuter le plan” (Execute plan). The latter is grayed out if the plan has nothing to do, or until the irreversibility is acknowledged.

4. Execution

The screen shows “Exécution en cours” (Execution in progress), then “N / M opérations traitées” (N / M operations processed) and a percentage as soon as the total is known (before that: “Préparation : lecture du plan…” (Preparing: reading the plan…)).

For each file, the app:

1. writes the copy under a temporary .obscura-part name on the destination disk;
2. computes the SHA-256 fingerprint of the bytes read during the copy, then rereads the copy from disk to verify it;
3. gives the copy its final name only once it is verified;
4. for a move, removes the source (Trash or permanent deletion) only after all copies of its unit have been verified, and only if the source is still the one that was copied: same size, same modification date, same file on the same disk. Before a permanent deletion, the source is also read again in full and compared with its copy’s fingerprint.

Each step is written to the journal before it is performed.

Before the first write, the app checks that the disk mounted at the destination is the one in the plan (its volume identifier), not merely a disk with the same name: otherwise the transfer stops, “Volume indisponible” (Volume unavailable), without writing anything.

If a file in a unit cannot receive its final name (a file with the same name has appeared in the meantime, for example), the whole unit is undone — the RAW is never left without its XMP — and the transfer continues with the next files.

If a source has been modified since its copy (a program rewrote a tag, for example), or if its copy has disappeared from the destination, the source is not removed: it stays in place, the Activité (Activity) screen reports it (“source(s) conservée(s) : retrait refusé” (source(s) kept: removal refused)), and the transfer continues.

There is no pause or cancel button for a transfer in the current version. During execution:

• quitting (⌘Q, “Quitter” (Quit) menu) opens an alert: “Quitter interrompra le travail en cours” (Quitting will interrupt the work in progress), with the number of operations already processed. The default button is “Annuler” (Cancel) (the transfer continues); “Quitter quand même” (Quit anyway) interrupts it — nothing is lost, it will be offered for resuming at the next launch;
• closing the window does not quit the app: the transfer continues without a window. Reopen it by clicking the Dock icon;
• the Mac’s idle sleep is held off until the end of the transfer. Closing the lid or putting the Mac to sleep yourself remains possible: the transfer is then interrupted, and can be resumed.

5. Interruption and resume

If the destination disk fills up or a volume disappears (disk unplugged), the transfer stops cleanly. The screen shows:

• “Disque de destination plein” (Destination disk full), “Volume indisponible” (Volume unavailable) or “Transfert interrompu” (Transfer interrupted);
• “Transfert interrompu (cause) : détail” (Transfer interrupted (cause): detail);
• “Rien ne reprend sans vous. La reprise lit le journal, défait l’opération restée ouverte, puis relance le plan : ce qui est déjà vérifié n’est pas recopié.” (Nothing resumes without you. Resuming reads the journal, undoes the operation left open, then restarts the plan: what is already verified is not copied again.)

Free up space or plug the disk back in, then click “Reprendre” (Resume) (the screen shows “Reprise en cours” (Resuming)). Or “Revenir aux options” (Back to options): nothing is erased, the journal keeps the open operation.

Resuming touches only the resumed transfer, never copies made since by another plan. If the source disk is not connected at the time of the resume, a source removal left pending is not considered done: it will be redone when the disk is reconnected and the transfer restarted.

The Centre de transfert open on a transfer interrupted during a previous session: nothing resumes before “Reprendre” (Resume).

If the source disk is not connected when you click “Reprendre” (Resume), nothing is read or written: the transfer stays interrupted (“Volume indisponible” (Volume unavailable)) and is still offered; reconnect the disk, then “Reprendre” again.

After the app is relaunched, an interrupted transfer is flagged from the home screen — the Tableau de bord (Dashboard) shows “1 transfert à reprendre” (1 transfer to resume) and an “Ouvrir le Centre de transfert” (Open the Transfer Center) button — as well as by the Activité (Activity) screen (“À reprendre” (To resume)). The Centre de transfert opens on it: “Transfert interrompu lors d’une session précédente” (Transfer interrupted during a previous session), the destination (“Destination : …” (Destination: …)) and the plan’s counts (“9 opération(s) au plan · 9 copie(s)” (9 operation(s) in the plan · 9 copy(ies))), then “Reprendre” (Resume). If the source disk is not connected, the header reads “Chemin inconnu : le volume de la source n’est pas joignable” (Unknown path: the source’s volume is unreachable). Details in chapter 10.

6. The report

The report of a transfer resumed after an interruption: 13 files transferred and verified, 1 already verified before the failure and not copied again, the manifest at the destination.

The “Rapport” (Report) card gives:

Then the space measures, following the same rules as the confirmation.

“Fichiers non consolidés” (Files not consolidated): each failed file, by its path in the source, with its cause in plain language — for example “lecture refusée (permissions du fichier ou du dossier)” (read refused (file or folder permissions)) for a file that macOS does not allow to be read, “source modifiée ou disparue depuis l’analyse” (source modified or gone since the analysis), “nom de destination déjà occupé” (destination name already taken), “annulé avec son groupe (fichier lié en échec)” (canceled with its group (linked file failed)) for the XMP of a RAW that could not be copied — and an “Afficher dans le Finder” (Show in Finder) button. The first hundred are named; beyond that, “… et N autres : le manifeste et l’Activité les nomment” (… and N others: the manifest and the Activity screen name them). The Activité (Activity) screen also names the first three (“1 en échec (illisible.jpg (SOURCE_PERMISSION_DENIED))” (1 failed (illisible.jpg (SOURCE_PERMISSION_DENIED)))).

“Sources conservées faute de Corbeille — à supprimer manuellement” (Sources kept for lack of a Trash — to delete manually): on a disk without a Trash (read-only, some formats), a move cannot put the source in the Trash. The app does not delete it for all that: the source is kept intact and listed here. The operation becomes a plain copy.

“Nouveau transfert” (New transfer) returns to the options.

The manifest

Every transfer executed from the app writes a manifest in the destination folder:

<destination>/.obscura-manifests/<identifiant du plan>/manifest.json

(<identifiant du plan> is the plan identifier.)

The report shows “Manifeste : chemin” (Manifest: path) and the “Afficher dans le Finder” (Show in Finder) button. The manifest is a readable JSON file that describes the transfer: the app that produced it and its version, the date, the source, the destination, the mode, and for each file its original path, its destination path, the operation, its status, its SHA-256 fingerprint, its size, the verification date and what happened to the source. A file that was not transferred carries its exact reason: SOURCE_PERMISSION_DENIED (read refused), SOURCE_DRIFT (source modified or gone since the analysis), DESTINATION_OCCUPIED, GROUP_ABORTED… It lets you find out, years later, where each file in the archive came from. It is written before the transfer is marked complete. The .obscura-manifests folder is ignored by analyses if the destination ever becomes a source again.