Skip to main content

SyncFolderPushCommand

The SyncFolderPushCommand is a high-level command designed to efficiently synchronize a local directory with a remote directory in your DID Space. It intelligently compares the local and remote states, uploading new or modified files and, optionally, deleting remote files that no longer exist locally. This command simplifies common tasks like deploying static websites, backing up important data, or distributing digital assets.

It handles complexities like concurrency, retries, and consistency checks, providing a robust and straightforward way to manage folder synchronization.

Use Cases

  • Deploying Static Websites: Push your local build directory (e.g., ./dist, ./build) to a public folder in your DID Space to deploy or update a website.
  • Data Backup: Regularly sync a local folder containing important documents or user data to your DID Space for a secure backup.
  • Asset Distribution: Upload a folder of images, videos, or other assets that your application needs to serve to users.

Input Parameters

The SyncFolderPushCommand accepts the following parameters in its input object.

  • source string | SourceObject[] (required) — Specifies the local source to sync from. This can be a string representing a local directory path, or an array of objects for more granular control.
  • target string (required) — The destination path in your DID Space where the files will be synced. It should end with a '/'.
  • metadata ObjectMetadata — Custom metadata to attach to the target folder object.
  • preview PreviewTemplate — A preview template to set for the target folder. See the guide on Managing Folder Previews for more details.
  • strictSync boolean (default: false) — If true, files that exist in the remote target but not in the local source will be deleted. Use with caution.
  • filter (object: Object) => boolean — A function to filter which local objects should be included in the synchronization. Return true to include the object, false to exclude it.
  • onProgress (progress: OnProgressInput) => void — A callback function that is triggered periodically to report the synchronization progress.
  • onAfterUpload (result: OnAfterUploadInput) => void — A callback function that is triggered after each individual file upload is completed.
  • concurrency number (default: 4) — The number of files to upload in parallel. The maximum is the number of CPU cores.
  • retryCount number (default: 3) — The number of times to retry a failed upload for a single file. The maximum is 10.
  • tmpDir string — Path to a temporary directory for processing files. Defaults to a system-generated temporary directory.
  • debug boolean (default: false) — If true, enables detailed debug logging to the console.

Output

The command returns a Promise that resolves to the following object.

  • statusCode number — The HTTP status code of the overall operation. 200 indicates success.
  • statusMessage string — A message providing more details about the outcome of the operation.
  • data object — An object containing the results of the synchronization process.
  • size number — The total size of all successfully uploaded files, in bytes.
  • count number — The total number of operations attempted (uploads and deletes).
  • errorCount number — The number of operations that failed.
  • duration number — The total time taken for the synchronization, in seconds.

Example

This example demonstrates how to sync a local folder named ./my-blog-dist to the /blog/ directory in a DID Space.

Syncing a Local Folder

typescript
import { SpaceClient, SyncFolderPushCommand } from '@blocklet/did-space-js';
import getWallet from '@blocklet/sdk/lib/wallet';
import path from 'path';
import fs from 'fs-extra';

// Ensure the source directory and some files exist for the example
const sourceDir = path.join(__dirname, 'my-blog-dist');
fs.ensureDirSync(sourceDir);
fs.writeFileSync(path.join(sourceDir, 'index.html'), '<h1>Hello World</h1>');
fs.ensureDirSync(path.join(sourceDir, 'assets'));
fs.writeFileSync(path.join(sourceDir, 'assets', 'style.css'), 'body { color: #333; }');

async function syncWebsite() {
  const wallet = getWallet();
  const client = new SpaceClient({
    endpoint: 'https://www.didspaces.com/app/api/space/...',
    wallet,
  });

  const command = new SyncFolderPushCommand({
    source: sourceDir,
    target: '/blog/',
    strictSync: true, // Delete remote files not in the source
    onProgress: ({ completed, total }) => {
      const percentage = total > 0 ? ((completed / total) * 100).toFixed(2) : 0;
      console.log(`Sync progress: ${completed}/${total} files (${percentage}%)`);
    },
  });

  const output = await client.send(command);

  if (output.statusCode === 200) {
    console.log('Sync completed successfully!');
    console.log(`Total files processed: ${output.data.count}`);
    console.log(`Errors: ${output.data.errorCount}`);
    console.log(`Duration: ${output.data.duration}s`);
  } else {
    console.error('Sync failed:', output.statusMessage);
  }

  // Clean up the created directory
  fs.removeSync(sourceDir);
}

syncWebsite();

Example Response (Success)

Response Data

json
{
  "statusCode": 200,
  "data": {
    "size": 48,
    "errorCount": 0,
    "count": 2,
    "duration": 1.25
  }
}

Best Practices

Ensuring Data Consistency

When syncing files, especially from a directory that might be actively written to, there's a risk of uploading an incomplete or corrupted file. To prevent this, the SyncFolderPushCommand first copies each file to a temporary directory before uploading. This ensures that a stable snapshot of the file is used for the upload, safeguarding against Unexpected end of form errors and ensuring data integrity.

Using strictSync Safely

The strictSync: true option is powerful for ensuring the remote directory is an exact mirror of the local source. However, it is a destructive operation. Any file on the remote server that is not present in the local source directory will be permanently deleted. Always double-check your source and target paths before enabling this option.

Custom Filtering

You can use the filter function to gain fine-grained control over which files are synchronized. For example, you can exclude dotfiles, log files, or specific directories.

Filtering Files

typescript
const command = new SyncFolderPushCommand({
  source: './project',
  target: '/my-project/',
  filter: (object) => {
    // Exclude all files starting with a dot (e.g., .DS_Store, .env)
    if (object.name.startsWith('.')) {
      return false;
    }
    // Exclude the node_modules directory
    if (object.key.includes('/node_modules/')) {
      return false;
    }
    return true;
  },
});